ShouldGCWals/ParticipantCopyITest.TestCopyParticipantOps/0: Unrecognized error type. Please see the error log for more information.

[==========] Running 25 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 21 tests from TxnParticipantITest
[ RUN      ] TxnParticipantITest.TestReplicateParticipantOps
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20240426 13:11:42.085028   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:38043
I20240426 13:11:42.085829   398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20240426 13:11:42.086337   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:11:42.093037   404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:42.093859   398 server_base.cc:1034] running on GCE node
W20240426 13:11:42.094558   406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:42.095324   403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:42.095675   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:11:42.095746   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:11:42.095772   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137102095770 us; error 0 us; skew 500 ppm
I20240426 13:11:42.101493   398 webserver.cc:460] Webserver started at http://127.0.99.190:33757/ using document root <none> and password file <none>
I20240426 13:11:42.101943   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:11:42.102072   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:11:42.102257   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:11:42.103768   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "22979287850c4c6ba3bd78b2edc28ab1"
format_stamp: "Formatted at 2024-04-26 13:11:42 on dist-test-slave-2jsl"
I20240426 13:11:42.107245   398 fs_manager.cc:691] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.004s
I20240426 13:11:42.109045   411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.109997   398 fs_manager.cc:725] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:11:42.110080   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "22979287850c4c6ba3bd78b2edc28ab1"
format_stamp: "Formatted at 2024-04-26 13:11:42 on dist-test-slave-2jsl"
I20240426 13:11:42.110153   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:11:42.128865   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:11:42.129272   398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20240426 13:11:42.129419   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:11:42.135421   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:38043
I20240426 13:11:42.135428   463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:38043 every 8 connection(s)
I20240426 13:11:42.137549   464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:11:42.141829   464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1: Bootstrap starting.
I20240426 13:11:42.143776   464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1: Neither blocks nor log segments found. Creating new log.
I20240426 13:11:42.144285   464 log.cc:824] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1: Log is configured to *not* fsync() on all Append() calls
I20240426 13:11:42.145444   464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1: No bootstrap required, opened a new log
I20240426 13:11:42.147485   464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22979287850c4c6ba3bd78b2edc28ab1" member_type: VOTER }
I20240426 13:11:42.147603   464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:11:42.147641   464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22979287850c4c6ba3bd78b2edc28ab1, State: Initialized, Role: FOLLOWER
I20240426 13:11:42.147976   464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22979287850c4c6ba3bd78b2edc28ab1" member_type: VOTER }
I20240426 13:11:42.148080   464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:11:42.148129   464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:11:42.148216   464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:11:42.148770   464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22979287850c4c6ba3bd78b2edc28ab1" member_type: VOTER }
I20240426 13:11:42.149024   464 leader_election.cc:302] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22979287850c4c6ba3bd78b2edc28ab1; no voters: 
I20240426 13:11:42.149251   464 leader_election.cc:288] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:11:42.149334   467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:11:42.149601   467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 1 LEADER]: Becoming Leader. State: Replica: 22979287850c4c6ba3bd78b2edc28ab1, State: Running, Role: LEADER
I20240426 13:11:42.149868   467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22979287850c4c6ba3bd78b2edc28ab1" member_type: VOTER }
I20240426 13:11:42.150261   464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:11:42.151268   468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22979287850c4c6ba3bd78b2edc28ab1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22979287850c4c6ba3bd78b2edc28ab1" member_type: VOTER } }
I20240426 13:11:42.151396   468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [sys.catalog]: This master's current role is: LEADER
I20240426 13:11:42.151731   469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22979287850c4c6ba3bd78b2edc28ab1. Latest consensus state: current_term: 1 leader_uuid: "22979287850c4c6ba3bd78b2edc28ab1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22979287850c4c6ba3bd78b2edc28ab1" member_type: VOTER } }
I20240426 13:11:42.151875   469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [sys.catalog]: This master's current role is: LEADER
I20240426 13:11:42.152082   476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:11:42.154773   476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:11:42.155068   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:11:42.158604   476 catalog_manager.cc:1347] Generated new cluster ID: b014bc4906e24d48a236dbe6a5dddb2b
I20240426 13:11:42.158661   476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:11:42.172324   476 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:11:42.173295   476 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:11:42.179222   476 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1: Generated new TSK 0
I20240426 13:11:42.179834   476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:11:42.187332   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:11:42.190388   486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:42.191191   487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:42.191900   489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:42.192586   398 server_base.cc:1034] running on GCE node
I20240426 13:11:42.192749   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:11:42.192790   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:11:42.192809   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137102192809 us; error 0 us; skew 500 ppm
I20240426 13:11:42.193656   398 webserver.cc:460] Webserver started at http://127.0.99.129:37317/ using document root <none> and password file <none>
I20240426 13:11:42.193789   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:11:42.193833   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:11:42.193894   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:11:42.194245   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "91f97cd273d243208e943e62664642ab"
format_stamp: "Formatted at 2024-04-26 13:11:42 on dist-test-slave-2jsl"
I20240426 13:11:42.195775   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:11:42.196686   494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.196969   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:11:42.197039   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "91f97cd273d243208e943e62664642ab"
format_stamp: "Formatted at 2024-04-26 13:11:42 on dist-test-slave-2jsl"
I20240426 13:11:42.197099   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:11:42.212682   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:11:42.212956   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:11:42.213382   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:11:42.214076   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:11:42.214126   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.214174   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:11:42.214205   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.219092   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:39325
I20240426 13:11:42.219161   557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:39325 every 8 connection(s)
I20240426 13:11:42.219921   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:11:42.221993   565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:42.223196   563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:42.223853   398 server_base.cc:1034] running on GCE node
W20240426 13:11:42.227137   560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:42.227371   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:11:42.227414   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:11:42.227432   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137102227432 us; error 0 us; skew 500 ppm
I20240426 13:11:42.228284   398 webserver.cc:460] Webserver started at http://127.0.99.130:33691/ using document root <none> and password file <none>
I20240426 13:11:42.228435   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:11:42.228477   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:11:42.228569   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:11:42.228910   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "7cd5da0b324e42de80b10d4d970fb014"
format_stamp: "Formatted at 2024-04-26 13:11:42 on dist-test-slave-2jsl"
I20240426 13:11:42.230522   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:11:42.231341   570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.231624   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:42.231738   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "7cd5da0b324e42de80b10d4d970fb014"
format_stamp: "Formatted at 2024-04-26 13:11:42 on dist-test-slave-2jsl"
I20240426 13:11:42.231812   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:11:42.233086   558 heartbeater.cc:347] Connected to a master server at 127.0.99.190:38043
I20240426 13:11:42.233249   558 heartbeater.cc:464] Registering TS with master...
I20240426 13:11:42.233573   558 heartbeater.cc:511] Master 127.0.99.190:38043 requested a full tablet report, sending...
I20240426 13:11:42.235235   428 ts_manager.cc:194] Registered new tserver with Master: 91f97cd273d243208e943e62664642ab (127.0.99.129:39325)
I20240426 13:11:42.236877   428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39258
I20240426 13:11:42.242486   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:11:42.242758   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:11:42.242995   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:11:42.243350   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:11:42.243384   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.243424   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:11:42.243453   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.247972   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:40563
I20240426 13:11:42.248013   633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:40563 every 8 connection(s)
I20240426 13:11:42.248685   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:11:42.250734   637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:42.252447   638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:42.252488   634 heartbeater.cc:347] Connected to a master server at 127.0.99.190:38043
I20240426 13:11:42.252543   634 heartbeater.cc:464] Registering TS with master...
I20240426 13:11:42.252712   634 heartbeater.cc:511] Master 127.0.99.190:38043 requested a full tablet report, sending...
I20240426 13:11:42.252888   398 server_base.cc:1034] running on GCE node
I20240426 13:11:42.253240   428 ts_manager.cc:194] Registered new tserver with Master: 7cd5da0b324e42de80b10d4d970fb014 (127.0.99.130:40563)
I20240426 13:11:42.253643   428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39260
W20240426 13:11:42.254132   640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:42.254372   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:11:42.254415   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:11:42.254427   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137102254427 us; error 0 us; skew 500 ppm
I20240426 13:11:42.255154   398 webserver.cc:460] Webserver started at http://127.0.99.131:34623/ using document root <none> and password file <none>
I20240426 13:11:42.255271   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:11:42.255312   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:11:42.255362   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:11:42.255647   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "55152695fd004fd9952601e3dced5e26"
format_stamp: "Formatted at 2024-04-26 13:11:42 on dist-test-slave-2jsl"
I20240426 13:11:42.256860   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:42.257601   645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.257818   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:42.257887   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "55152695fd004fd9952601e3dced5e26"
format_stamp: "Formatted at 2024-04-26 13:11:42 on dist-test-slave-2jsl"
I20240426 13:11:42.257961   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestReplicateParticipantOps.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:11:42.279659   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:11:42.279953   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:11:42.280180   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:11:42.280560   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:11:42.280596   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.280635   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:11:42.280669   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:42.286164   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:34331
I20240426 13:11:42.286231   708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:34331 every 8 connection(s)
I20240426 13:11:42.290184   709 heartbeater.cc:347] Connected to a master server at 127.0.99.190:38043
I20240426 13:11:42.290261   709 heartbeater.cc:464] Registering TS with master...
I20240426 13:11:42.290414   709 heartbeater.cc:511] Master 127.0.99.190:38043 requested a full tablet report, sending...
I20240426 13:11:42.290927   428 ts_manager.cc:194] Registered new tserver with Master: 55152695fd004fd9952601e3dced5e26 (127.0.99.131:34331)
I20240426 13:11:42.291278   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004709776s
I20240426 13:11:42.291460   428 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39276
I20240426 13:11:42.291939   398 test_util.cc:268] Using random seed: -1525410848
I20240426 13:11:42.303478   428 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39284:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:11:42.304625   428 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:11:42.316922   598 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba9a1671883405b8488f37066205126 (DEFAULT_TABLE table=test-workload [id=8474694846e84a589c7fa039a96ef1e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:11:42.317276   598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba9a1671883405b8488f37066205126. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:11:42.317750   673 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba9a1671883405b8488f37066205126 (DEFAULT_TABLE table=test-workload [id=8474694846e84a589c7fa039a96ef1e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:11:42.317968   673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba9a1671883405b8488f37066205126. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:11:42.319046   724 tablet_bootstrap.cc:492] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014: Bootstrap starting.
I20240426 13:11:42.319135   522 tablet_service.cc:1467] Processing CreateTablet for tablet 8ba9a1671883405b8488f37066205126 (DEFAULT_TABLE table=test-workload [id=8474694846e84a589c7fa039a96ef1e9]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:11:42.319329   522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ba9a1671883405b8488f37066205126. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:11:42.320003   724 tablet_bootstrap.cc:654] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014: Neither blocks nor log segments found. Creating new log.
I20240426 13:11:42.320026   725 tablet_bootstrap.cc:492] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26: Bootstrap starting.
I20240426 13:11:42.320644   725 tablet_bootstrap.cc:654] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26: Neither blocks nor log segments found. Creating new log.
I20240426 13:11:42.320920   724 tablet_bootstrap.cc:492] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014: No bootstrap required, opened a new log
I20240426 13:11:42.321017   724 ts_tablet_manager.cc:1367] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:11:42.321106   727 tablet_bootstrap.cc:492] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab: Bootstrap starting.
I20240426 13:11:42.321444   724 raft_consensus.cc:348] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.321561   725 tablet_bootstrap.cc:492] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26: No bootstrap required, opened a new log
I20240426 13:11:42.321646   725 ts_tablet_manager.cc:1367] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:11:42.321873   727 tablet_bootstrap.cc:654] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab: Neither blocks nor log segments found. Creating new log.
I20240426 13:11:42.321916   725 raft_consensus.cc:348] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.322021   725 raft_consensus.cc:374] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:11:42.322103   725 raft_consensus.cc:729] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55152695fd004fd9952601e3dced5e26, State: Initialized, Role: FOLLOWER
I20240426 13:11:42.321561   724 raft_consensus.cc:374] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:11:42.322211   724 raft_consensus.cc:729] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd5da0b324e42de80b10d4d970fb014, State: Initialized, Role: FOLLOWER
I20240426 13:11:42.322227   725 consensus_queue.cc:260] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.322324   724 consensus_queue.cc:260] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.322568   725 ts_tablet_manager.cc:1398] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:42.322603   724 ts_tablet_manager.cc:1398] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:11:42.322661   709 heartbeater.cc:503] Master 127.0.99.190:38043 was elected leader, sending a full tablet report...
I20240426 13:11:42.322727   634 heartbeater.cc:503] Master 127.0.99.190:38043 was elected leader, sending a full tablet report...
I20240426 13:11:42.323447   727 tablet_bootstrap.cc:492] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab: No bootstrap required, opened a new log
I20240426 13:11:42.323544   727 ts_tablet_manager.cc:1367] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab: Time spent bootstrapping tablet: real 0.003s	user 0.000s	sys 0.002s
I20240426 13:11:42.323828   727 raft_consensus.cc:348] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.323956   727 raft_consensus.cc:374] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:11:42.323997   727 raft_consensus.cc:729] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91f97cd273d243208e943e62664642ab, State: Initialized, Role: FOLLOWER
I20240426 13:11:42.324115   727 consensus_queue.cc:260] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.324386   727 ts_tablet_manager.cc:1398] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:11:42.324510   558 heartbeater.cc:503] Master 127.0.99.190:38043 was elected leader, sending a full tablet report...
W20240426 13:11:42.469843   559 tablet.cc:2369] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 13:11:42.489701   732 raft_consensus.cc:482] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:11:42.489790   732 raft_consensus.cc:504] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.490470   732 leader_election.cc:288] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cd5da0b324e42de80b10d4d970fb014 (127.0.99.130:40563), 55152695fd004fd9952601e3dced5e26 (127.0.99.131:34331)
I20240426 13:11:42.493521   608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba9a1671883405b8488f37066205126" candidate_uuid: "91f97cd273d243208e943e62664642ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd5da0b324e42de80b10d4d970fb014" is_pre_election: true
I20240426 13:11:42.493682   682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba9a1671883405b8488f37066205126" candidate_uuid: "91f97cd273d243208e943e62664642ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55152695fd004fd9952601e3dced5e26" is_pre_election: true
I20240426 13:11:42.493708   608 raft_consensus.cc:2454] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f97cd273d243208e943e62664642ab in term 0.
I20240426 13:11:42.493854   682 raft_consensus.cc:2454] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91f97cd273d243208e943e62664642ab in term 0.
I20240426 13:11:42.494184   495 leader_election.cc:302] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55152695fd004fd9952601e3dced5e26, 91f97cd273d243208e943e62664642ab; no voters: 
I20240426 13:11:42.494326   732 raft_consensus.cc:2781] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:11:42.494380   732 raft_consensus.cc:482] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:11:42.494421   732 raft_consensus.cc:3037] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:11:42.495338   732 raft_consensus.cc:504] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.495625   732 leader_election.cc:288] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [CANDIDATE]: Term 1 election: Requested vote from peers 7cd5da0b324e42de80b10d4d970fb014 (127.0.99.130:40563), 55152695fd004fd9952601e3dced5e26 (127.0.99.131:34331)
I20240426 13:11:42.495891   608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba9a1671883405b8488f37066205126" candidate_uuid: "91f97cd273d243208e943e62664642ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd5da0b324e42de80b10d4d970fb014"
I20240426 13:11:42.495978   608 raft_consensus.cc:3037] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:11:42.495963   682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ba9a1671883405b8488f37066205126" candidate_uuid: "91f97cd273d243208e943e62664642ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55152695fd004fd9952601e3dced5e26"
I20240426 13:11:42.496098   682 raft_consensus.cc:3037] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:11:42.496711   608 raft_consensus.cc:2454] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f97cd273d243208e943e62664642ab in term 1.
I20240426 13:11:42.496752   682 raft_consensus.cc:2454] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91f97cd273d243208e943e62664642ab in term 1.
I20240426 13:11:42.496953   495 leader_election.cc:302] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd5da0b324e42de80b10d4d970fb014, 91f97cd273d243208e943e62664642ab; no voters: 
I20240426 13:11:42.497098   732 raft_consensus.cc:2781] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:11:42.497390   732 raft_consensus.cc:686] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 1 LEADER]: Becoming Leader. State: Replica: 91f97cd273d243208e943e62664642ab, State: Running, Role: LEADER
I20240426 13:11:42.497560   732 consensus_queue.cc:237] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } }
I20240426 13:11:42.499298   427 catalog_manager.cc:5484] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 91f97cd273d243208e943e62664642ab (127.0.99.129). New cstate: current_term: 1 leader_uuid: "91f97cd273d243208e943e62664642ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91f97cd273d243208e943e62664642ab" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 39325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 } health_report { overall_health: UNKNOWN } } }
I20240426 13:11:42.564842   682 raft_consensus.cc:1261] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 1 FOLLOWER]: Refusing update from remote peer 91f97cd273d243208e943e62664642ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:11:42.564842   608 raft_consensus.cc:1261] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 1 FOLLOWER]: Refusing update from remote peer 91f97cd273d243208e943e62664642ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:11:42.565318   734 consensus_queue.cc:1035] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "55152695fd004fd9952601e3dced5e26" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:11:42.565466   732 consensus_queue.cc:1035] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd5da0b324e42de80b10d4d970fb014" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:11:42.568478   741 mvcc.cc:205] Tried to move back new op lower bound from 7021105572104704000 to 7021105571832225792. Current Snapshot: MvccSnapshot[applied={T|T < 7021105572101730304}]
I20240426 13:11:52.582839   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:11:52.587330   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:11:52.587482   398 tablet_replica.cc:330] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab: stopping tablet replica
I20240426 13:11:52.587666   398 raft_consensus.cc:2229] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:11:52.587848   398 raft_consensus.cc:2258] T 8ba9a1671883405b8488f37066205126 P 91f97cd273d243208e943e62664642ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:11:52.605619   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:11:52.608680   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:11:52.611886   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:11:52.612035   398 tablet_replica.cc:330] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014: stopping tablet replica
I20240426 13:11:52.612146   398 raft_consensus.cc:2229] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:11:52.612248   398 raft_consensus.cc:2258] T 8ba9a1671883405b8488f37066205126 P 7cd5da0b324e42de80b10d4d970fb014 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:11:52.628665   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:11:52.630029   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:11:52.633466   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:11:52.633605   398 tablet_replica.cc:330] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26: stopping tablet replica
I20240426 13:11:52.633695   398 raft_consensus.cc:2229] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:11:52.633780   398 raft_consensus.cc:2258] T 8ba9a1671883405b8488f37066205126 P 55152695fd004fd9952601e3dced5e26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:11:52.638739   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:11:52.640122   398 master.cc:537] Master@127.0.99.190:38043 shutting down...
I20240426 13:11:52.643241   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:11:52.643342   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:11:52.643404   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 22979287850c4c6ba3bd78b2edc28ab1: stopping tablet replica
I20240426 13:11:52.655134   398 master.cc:552] Master@127.0.99.190:38043 shutdown complete.
[       OK ] TxnParticipantITest.TestReplicateParticipantOps (10579 ms)
[ RUN      ] TxnParticipantITest.TestWaitOnFinalizeCommit
I20240426 13:11:52.663689   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:45315
I20240426 13:11:52.663929   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:11:52.666348   810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:52.667160   807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:52.667940   808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:52.668709   398 server_base.cc:1034] running on GCE node
I20240426 13:11:52.668839   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:11:52.668869   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:11:52.668887   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137112668886 us; error 0 us; skew 500 ppm
I20240426 13:11:52.669723   398 webserver.cc:460] Webserver started at http://127.0.99.190:42009/ using document root <none> and password file <none>
I20240426 13:11:52.669884   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:11:52.669945   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:11:52.670043   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:11:52.670424   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "f70c6815d05547fd9590c12ac5bd565a"
format_stamp: "Formatted at 2024-04-26 13:11:52 on dist-test-slave-2jsl"
I20240426 13:11:52.671841   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:11:52.672770   815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.672991   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:52.673059   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "f70c6815d05547fd9590c12ac5bd565a"
format_stamp: "Formatted at 2024-04-26 13:11:52 on dist-test-slave-2jsl"
I20240426 13:11:52.673166   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:11:52.690515   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:11:52.690770   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:11:52.694459   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:45315
I20240426 13:11:52.696324   868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:11:52.700139   867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:45315 every 8 connection(s)
I20240426 13:11:52.701093   868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a: Bootstrap starting.
I20240426 13:11:52.701759   868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a: Neither blocks nor log segments found. Creating new log.
I20240426 13:11:52.702621   868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a: No bootstrap required, opened a new log
I20240426 13:11:52.702934   868 raft_consensus.cc:348] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70c6815d05547fd9590c12ac5bd565a" member_type: VOTER }
I20240426 13:11:52.703012   868 raft_consensus.cc:374] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:11:52.703047   868 raft_consensus.cc:729] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70c6815d05547fd9590c12ac5bd565a, State: Initialized, Role: FOLLOWER
I20240426 13:11:52.703263   868 consensus_queue.cc:260] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70c6815d05547fd9590c12ac5bd565a" member_type: VOTER }
I20240426 13:11:52.703384   868 raft_consensus.cc:388] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:11:52.703466   868 raft_consensus.cc:482] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:11:52.703518   868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:11:52.704077   868 raft_consensus.cc:504] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70c6815d05547fd9590c12ac5bd565a" member_type: VOTER }
I20240426 13:11:52.704182   868 leader_election.cc:302] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f70c6815d05547fd9590c12ac5bd565a; no voters: 
I20240426 13:11:52.704353   868 leader_election.cc:288] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:11:52.704447   871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:11:52.704653   871 raft_consensus.cc:686] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 1 LEADER]: Becoming Leader. State: Replica: f70c6815d05547fd9590c12ac5bd565a, State: Running, Role: LEADER
I20240426 13:11:52.704708   868 sys_catalog.cc:564] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:11:52.704787   871 consensus_queue.cc:237] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70c6815d05547fd9590c12ac5bd565a" member_type: VOTER }
I20240426 13:11:52.705233   872 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f70c6815d05547fd9590c12ac5bd565a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70c6815d05547fd9590c12ac5bd565a" member_type: VOTER } }
I20240426 13:11:52.705274   873 sys_catalog.cc:455] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f70c6815d05547fd9590c12ac5bd565a. Latest consensus state: current_term: 1 leader_uuid: "f70c6815d05547fd9590c12ac5bd565a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f70c6815d05547fd9590c12ac5bd565a" member_type: VOTER } }
I20240426 13:11:52.705415   873 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [sys.catalog]: This master's current role is: LEADER
I20240426 13:11:52.705662   880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:11:52.705827   872 sys_catalog.cc:458] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [sys.catalog]: This master's current role is: LEADER
I20240426 13:11:52.706583   880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:11:52.706828   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:11:52.708189   880 catalog_manager.cc:1347] Generated new cluster ID: 8d8c888f3a9d442b9a867cda962236ab
I20240426 13:11:52.708235   880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:11:52.728955   880 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:11:52.729712   880 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:11:52.742141   880 catalog_manager.cc:5857] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a: Generated new TSK 0
I20240426 13:11:52.742303   880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:11:52.770933   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:11:52.773502   889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:52.774281   890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:52.775166   398 server_base.cc:1034] running on GCE node
W20240426 13:11:52.775805   892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:52.775985   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:11:52.776026   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:11:52.776038   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137112776038 us; error 0 us; skew 500 ppm
I20240426 13:11:52.776782   398 webserver.cc:460] Webserver started at http://127.0.99.129:41203/ using document root <none> and password file <none>
I20240426 13:11:52.776907   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:11:52.776947   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:11:52.776993   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:11:52.777323   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "dcd7ce5134174247b11a9cb44e8e4693"
format_stamp: "Formatted at 2024-04-26 13:11:52 on dist-test-slave-2jsl"
I20240426 13:11:52.778579   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:11:52.779402   897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.779621   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:52.779681   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "dcd7ce5134174247b11a9cb44e8e4693"
format_stamp: "Formatted at 2024-04-26 13:11:52 on dist-test-slave-2jsl"
I20240426 13:11:52.779742   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:11:52.792902   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:11:52.793210   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:11:52.793432   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:11:52.793780   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:11:52.793814   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.793853   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:11:52.793887   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.797726   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:43269
I20240426 13:11:52.797767   960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:43269 every 8 connection(s)
I20240426 13:11:52.798411   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:11:52.800719   965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:52.801468   966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:52.802183   961 heartbeater.cc:347] Connected to a master server at 127.0.99.190:45315
W20240426 13:11:52.802275   968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:52.802284   961 heartbeater.cc:464] Registering TS with master...
I20240426 13:11:52.802548   961 heartbeater.cc:511] Master 127.0.99.190:45315 requested a full tablet report, sending...
I20240426 13:11:52.803053   398 server_base.cc:1034] running on GCE node
I20240426 13:11:52.803210   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20240426 13:11:52.803193   832 ts_manager.cc:194] Registered new tserver with Master: dcd7ce5134174247b11a9cb44e8e4693 (127.0.99.129:43269)
W20240426 13:11:52.803261   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:11:52.803319   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137112803319 us; error 0 us; skew 500 ppm
I20240426 13:11:52.803886   832 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45542
I20240426 13:11:52.804147   398 webserver.cc:460] Webserver started at http://127.0.99.130:39255/ using document root <none> and password file <none>
I20240426 13:11:52.804275   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:11:52.804317   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:11:52.804368   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:11:52.804947   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "b2071c8ff8534b9eb48d95de3b3453de"
format_stamp: "Formatted at 2024-04-26 13:11:52 on dist-test-slave-2jsl"
I20240426 13:11:52.806579   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:11:52.807334   973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.807561   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:52.807624   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "b2071c8ff8534b9eb48d95de3b3453de"
format_stamp: "Formatted at 2024-04-26 13:11:52 on dist-test-slave-2jsl"
I20240426 13:11:52.807686   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:11:52.818419   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:11:52.818667   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:11:52.818871   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:11:52.819249   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:11:52.819284   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.819324   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:11:52.819352   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.823195   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:39405
I20240426 13:11:52.823364  1036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:39405 every 8 connection(s)
I20240426 13:11:52.823896   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:11:52.825857  1040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:52.826607  1043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:11:52.827406  1041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:11:52.827426  1037 heartbeater.cc:347] Connected to a master server at 127.0.99.190:45315
I20240426 13:11:52.827503  1037 heartbeater.cc:464] Registering TS with master...
I20240426 13:11:52.827673  1037 heartbeater.cc:511] Master 127.0.99.190:45315 requested a full tablet report, sending...
I20240426 13:11:52.828037   398 server_base.cc:1034] running on GCE node
I20240426 13:11:52.828207   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20240426 13:11:52.828248   832 ts_manager.cc:194] Registered new tserver with Master: b2071c8ff8534b9eb48d95de3b3453de (127.0.99.130:39405)
W20240426 13:11:52.828254   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:11:52.828326   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137112828326 us; error 0 us; skew 500 ppm
I20240426 13:11:52.828832   832 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45548
I20240426 13:11:52.829165   398 webserver.cc:460] Webserver started at http://127.0.99.131:43143/ using document root <none> and password file <none>
I20240426 13:11:52.829282   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:11:52.829331   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:11:52.829398   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:11:52.829710   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "e097ed19e4c0483b9713553277d9decb"
format_stamp: "Formatted at 2024-04-26 13:11:52 on dist-test-slave-2jsl"
I20240426 13:11:52.830974   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:11:52.831746  1048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.831991   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:52.832065   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "e097ed19e4c0483b9713553277d9decb"
format_stamp: "Formatted at 2024-04-26 13:11:52 on dist-test-slave-2jsl"
I20240426 13:11:52.832108   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnFinalizeCommit.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:11:52.865785   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:11:52.866070   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:11:52.866277   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:11:52.866624   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:11:52.866658   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.866684   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:11:52.866711   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:11:52.870576   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:41669
I20240426 13:11:52.870626  1111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:41669 every 8 connection(s)
I20240426 13:11:52.874270  1112 heartbeater.cc:347] Connected to a master server at 127.0.99.190:45315
I20240426 13:11:52.874334  1112 heartbeater.cc:464] Registering TS with master...
I20240426 13:11:52.874511  1112 heartbeater.cc:511] Master 127.0.99.190:45315 requested a full tablet report, sending...
I20240426 13:11:52.874930   832 ts_manager.cc:194] Registered new tserver with Master: e097ed19e4c0483b9713553277d9decb (127.0.99.131:41669)
I20240426 13:11:52.875370   832 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45552
I20240426 13:11:52.875399   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004468458s
I20240426 13:11:52.875828   398 test_util.cc:268] Using random seed: -1514826957
I20240426 13:11:52.882182   832 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45568:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:11:52.882503   832 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:11:52.889776   925 tablet_service.cc:1467] Processing CreateTablet for tablet 719f7cb09b4341bd8df65d6d24383c75 (DEFAULT_TABLE table=test-workload [id=10892b03b01049d3ab81ddf51a342a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:11:52.890040   925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 719f7cb09b4341bd8df65d6d24383c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:11:52.890933  1076 tablet_service.cc:1467] Processing CreateTablet for tablet 719f7cb09b4341bd8df65d6d24383c75 (DEFAULT_TABLE table=test-workload [id=10892b03b01049d3ab81ddf51a342a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:11:52.891018  1001 tablet_service.cc:1467] Processing CreateTablet for tablet 719f7cb09b4341bd8df65d6d24383c75 (DEFAULT_TABLE table=test-workload [id=10892b03b01049d3ab81ddf51a342a91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:11:52.891139  1076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 719f7cb09b4341bd8df65d6d24383c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:11:52.891194  1001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 719f7cb09b4341bd8df65d6d24383c75. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:11:52.892953  1128 tablet_bootstrap.cc:492] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb: Bootstrap starting.
I20240426 13:11:52.893033  1129 tablet_bootstrap.cc:492] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de: Bootstrap starting.
I20240426 13:11:52.893347  1127 tablet_bootstrap.cc:492] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693: Bootstrap starting.
I20240426 13:11:52.893828  1129 tablet_bootstrap.cc:654] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de: Neither blocks nor log segments found. Creating new log.
I20240426 13:11:52.893918  1128 tablet_bootstrap.cc:654] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb: Neither blocks nor log segments found. Creating new log.
I20240426 13:11:52.894157  1127 tablet_bootstrap.cc:654] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693: Neither blocks nor log segments found. Creating new log.
I20240426 13:11:52.894834  1129 tablet_bootstrap.cc:492] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de: No bootstrap required, opened a new log
I20240426 13:11:52.894917  1129 ts_tablet_manager.cc:1367] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de: Time spent bootstrapping tablet: real 0.002s	user 0.000s	sys 0.001s
I20240426 13:11:52.894938  1128 tablet_bootstrap.cc:492] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb: No bootstrap required, opened a new log
I20240426 13:11:52.895035  1128 ts_tablet_manager.cc:1367] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:11:52.895326  1129 raft_consensus.cc:348] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:52.895431  1129 raft_consensus.cc:374] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:11:52.895473  1129 raft_consensus.cc:729] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2071c8ff8534b9eb48d95de3b3453de, State: Initialized, Role: FOLLOWER
I20240426 13:11:52.895540  1128 raft_consensus.cc:348] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:52.895635  1128 raft_consensus.cc:374] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:11:52.895591  1129 consensus_queue.cc:260] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:52.895674  1128 raft_consensus.cc:729] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e097ed19e4c0483b9713553277d9decb, State: Initialized, Role: FOLLOWER
I20240426 13:11:52.895736  1127 tablet_bootstrap.cc:492] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693: No bootstrap required, opened a new log
I20240426 13:11:52.895800  1127 ts_tablet_manager.cc:1367] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:11:52.895799  1128 consensus_queue.cc:260] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:52.895912  1129 ts_tablet_manager.cc:1398] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:11:52.896039  1037 heartbeater.cc:503] Master 127.0.99.190:45315 was elected leader, sending a full tablet report...
I20240426 13:11:52.896132  1128 ts_tablet_manager.cc:1398] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:52.896184  1127 raft_consensus.cc:348] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:52.896286  1127 raft_consensus.cc:374] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:11:52.896301  1112 heartbeater.cc:503] Master 127.0.99.190:45315 was elected leader, sending a full tablet report...
I20240426 13:11:52.896317  1127 raft_consensus.cc:729] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcd7ce5134174247b11a9cb44e8e4693, State: Initialized, Role: FOLLOWER
I20240426 13:11:52.896453  1127 consensus_queue.cc:260] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:52.896749  1127 ts_tablet_manager.cc:1398] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:11:52.897008   961 heartbeater.cc:503] Master 127.0.99.190:45315 was elected leader, sending a full tablet report...
I20240426 13:11:53.241686  1133 raft_consensus.cc:482] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:11:53.241801  1133 raft_consensus.cc:504] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:53.242303  1133 leader_election.cc:288] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcd7ce5134174247b11a9cb44e8e4693 (127.0.99.129:43269), e097ed19e4c0483b9713553277d9decb (127.0.99.131:41669)
I20240426 13:11:53.245461  1086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719f7cb09b4341bd8df65d6d24383c75" candidate_uuid: "b2071c8ff8534b9eb48d95de3b3453de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e097ed19e4c0483b9713553277d9decb" is_pre_election: true
I20240426 13:11:53.245584   935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719f7cb09b4341bd8df65d6d24383c75" candidate_uuid: "b2071c8ff8534b9eb48d95de3b3453de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcd7ce5134174247b11a9cb44e8e4693" is_pre_election: true
I20240426 13:11:53.245615  1086 raft_consensus.cc:2454] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2071c8ff8534b9eb48d95de3b3453de in term 0.
I20240426 13:11:53.245718   935 raft_consensus.cc:2454] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b2071c8ff8534b9eb48d95de3b3453de in term 0.
I20240426 13:11:53.245934   976 leader_election.cc:302] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2071c8ff8534b9eb48d95de3b3453de, e097ed19e4c0483b9713553277d9decb; no voters: 
I20240426 13:11:53.246120  1133 raft_consensus.cc:2781] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:11:53.246181  1133 raft_consensus.cc:482] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:11:53.246229  1133 raft_consensus.cc:3037] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:11:53.247009  1133 raft_consensus.cc:504] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:53.247318  1133 leader_election.cc:288] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [CANDIDATE]: Term 1 election: Requested vote from peers dcd7ce5134174247b11a9cb44e8e4693 (127.0.99.129:43269), e097ed19e4c0483b9713553277d9decb (127.0.99.131:41669)
I20240426 13:11:53.247608  1086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719f7cb09b4341bd8df65d6d24383c75" candidate_uuid: "b2071c8ff8534b9eb48d95de3b3453de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e097ed19e4c0483b9713553277d9decb"
I20240426 13:11:53.247608   935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719f7cb09b4341bd8df65d6d24383c75" candidate_uuid: "b2071c8ff8534b9eb48d95de3b3453de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcd7ce5134174247b11a9cb44e8e4693"
I20240426 13:11:53.247737   935 raft_consensus.cc:3037] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:11:53.247737  1086 raft_consensus.cc:3037] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:11:53.248667   935 raft_consensus.cc:2454] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2071c8ff8534b9eb48d95de3b3453de in term 1.
I20240426 13:11:53.248667  1086 raft_consensus.cc:2454] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b2071c8ff8534b9eb48d95de3b3453de in term 1.
I20240426 13:11:53.248979   977 leader_election.cc:302] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b2071c8ff8534b9eb48d95de3b3453de, dcd7ce5134174247b11a9cb44e8e4693; no voters: 
I20240426 13:11:53.249405  1133 raft_consensus.cc:2781] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:11:53.249532  1133 raft_consensus.cc:686] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 1 LEADER]: Becoming Leader. State: Replica: b2071c8ff8534b9eb48d95de3b3453de, State: Running, Role: LEADER
I20240426 13:11:53.249644  1133 consensus_queue.cc:237] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:11:53.250943   830 catalog_manager.cc:5484] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de reported cstate change: term changed from 0 to 1, leader changed from <none> to b2071c8ff8534b9eb48d95de3b3453de (127.0.99.130). New cstate: current_term: 1 leader_uuid: "b2071c8ff8534b9eb48d95de3b3453de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } health_report { overall_health: HEALTHY } } }
I20240426 13:11:53.348017   935 raft_consensus.cc:1261] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 1 FOLLOWER]: Refusing update from remote peer b2071c8ff8534b9eb48d95de3b3453de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:11:53.348040  1086 raft_consensus.cc:1261] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 1 FOLLOWER]: Refusing update from remote peer b2071c8ff8534b9eb48d95de3b3453de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:11:53.348381  1133 consensus_queue.cc:1035] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:11:53.348498  1137 consensus_queue.cc:1035] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:03.840646  1170 raft_consensus.cc:979] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de: : Instructing follower dcd7ce5134174247b11a9cb44e8e4693 to start an election
I20240426 13:12:03.840745  1170 raft_consensus.cc:1067] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 1 LEADER]: Signalling peer dcd7ce5134174247b11a9cb44e8e4693 to start an election
I20240426 13:12:03.841188   935 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "719f7cb09b4341bd8df65d6d24383c75"
dest_uuid: "dcd7ce5134174247b11a9cb44e8e4693"
 from {username='slave'} at 127.0.0.1:35418
I20240426 13:12:03.841346   935 raft_consensus.cc:482] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:12:03.841408   935 raft_consensus.cc:3037] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:03.842371   935 raft_consensus.cc:504] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:12:03.842770   935 leader_election.cc:288] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [CANDIDATE]: Term 2 election: Requested vote from peers e097ed19e4c0483b9713553277d9decb (127.0.99.131:41669), b2071c8ff8534b9eb48d95de3b3453de (127.0.99.130:39405)
I20240426 13:12:03.845957  1085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719f7cb09b4341bd8df65d6d24383c75" candidate_uuid: "dcd7ce5134174247b11a9cb44e8e4693" candidate_term: 2 candidate_status { last_received { term: 1 index: 3078 } } ignore_live_leader: true dest_uuid: "e097ed19e4c0483b9713553277d9decb"
I20240426 13:12:03.845959  1011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "719f7cb09b4341bd8df65d6d24383c75" candidate_uuid: "dcd7ce5134174247b11a9cb44e8e4693" candidate_term: 2 candidate_status { last_received { term: 1 index: 3078 } } ignore_live_leader: true dest_uuid: "b2071c8ff8534b9eb48d95de3b3453de"
I20240426 13:12:03.846109  1085 raft_consensus.cc:3037] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:03.846128  1011 raft_consensus.cc:3032] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:12:03.846170  1011 raft_consensus.cc:729] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b2071c8ff8534b9eb48d95de3b3453de, State: Running, Role: LEADER
I20240426 13:12:03.846318  1011 consensus_queue.cc:260] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3078, Committed index: 3078, Last appended: 1.3078, Last appended by leader: 3078, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:12:03.846441  1011 raft_consensus.cc:3037] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:03.846849  1085 raft_consensus.cc:2454] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcd7ce5134174247b11a9cb44e8e4693 in term 2.
I20240426 13:12:03.847088  1011 raft_consensus.cc:2454] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcd7ce5134174247b11a9cb44e8e4693 in term 2.
I20240426 13:12:03.847110   900 leader_election.cc:302] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dcd7ce5134174247b11a9cb44e8e4693, e097ed19e4c0483b9713553277d9decb; no voters: 
I20240426 13:12:03.847352  1193 raft_consensus.cc:2781] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:12:03.847522  1193 raft_consensus.cc:686] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 2 LEADER]: Becoming Leader. State: Replica: dcd7ce5134174247b11a9cb44e8e4693, State: Running, Role: LEADER
I20240426 13:12:03.847644  1193 consensus_queue.cc:237] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3078, Committed index: 3078, Last appended: 1.3078, Last appended by leader: 3078, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } }
I20240426 13:12:03.848786   830 catalog_manager.cc:5484] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 reported cstate change: term changed from 1 to 2, leader changed from b2071c8ff8534b9eb48d95de3b3453de (127.0.99.130) to dcd7ce5134174247b11a9cb44e8e4693 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "dcd7ce5134174247b11a9cb44e8e4693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dcd7ce5134174247b11a9cb44e8e4693" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 } health_report { overall_health: UNKNOWN } } }
I20240426 13:12:03.850072  1085 raft_consensus.cc:1261] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 2 FOLLOWER]: Refusing update from remote peer dcd7ce5134174247b11a9cb44e8e4693: Log matching property violated. Preceding OpId in replica: term: 1 index: 3078. Preceding OpId from leader: term: 2 index: 3080. (index mismatch)
I20240426 13:12:03.850075  1011 raft_consensus.cc:1261] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 2 FOLLOWER]: Refusing update from remote peer dcd7ce5134174247b11a9cb44e8e4693: Log matching property violated. Preceding OpId in replica: term: 1 index: 3078. Preceding OpId from leader: term: 2 index: 3080. (index mismatch)
I20240426 13:12:03.850391  1193 consensus_queue.cc:1035] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2071c8ff8534b9eb48d95de3b3453de" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3079, Last known committed idx: 3078, Time since last communication: 0.000s
I20240426 13:12:03.850503  1194 consensus_queue.cc:1035] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e097ed19e4c0483b9713553277d9decb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3079, Last known committed idx: 3078, Time since last communication: 0.000s
I20240426 13:12:05.914777   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:12:05.918308   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:05.918464   398 tablet_replica.cc:330] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693: stopping tablet replica
I20240426 13:12:05.918561   398 raft_consensus.cc:2229] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:12:05.918701   398 raft_consensus.cc:2258] T 719f7cb09b4341bd8df65d6d24383c75 P dcd7ce5134174247b11a9cb44e8e4693 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:05.934114   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:12:05.935762   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:12:05.939396   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:05.939540   398 tablet_replica.cc:330] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de: stopping tablet replica
I20240426 13:12:05.939646   398 raft_consensus.cc:2229] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:05.939729   398 raft_consensus.cc:2258] T 719f7cb09b4341bd8df65d6d24383c75 P b2071c8ff8534b9eb48d95de3b3453de [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:05.955359   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:12:05.956705   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:12:05.960172   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:05.960412   398 tablet_replica.cc:330] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb: stopping tablet replica
I20240426 13:12:05.960518   398 raft_consensus.cc:2229] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:05.960600   398 raft_consensus.cc:2258] T 719f7cb09b4341bd8df65d6d24383c75 P e097ed19e4c0483b9713553277d9decb [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:05.975791   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:12:05.977169   398 master.cc:537] Master@127.0.99.190:45315 shutting down...
I20240426 13:12:05.979959   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:05.980063   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:05.980115   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P f70c6815d05547fd9590c12ac5bd565a: stopping tablet replica
I20240426 13:12:05.991768   398 master.cc:552] Master@127.0.99.190:45315 shutdown complete.
[       OK ] TxnParticipantITest.TestWaitOnFinalizeCommit (13335 ms)
[ RUN      ] TxnParticipantITest.TestWaitOnAbortCommit
I20240426 13:12:05.999598   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:38713
I20240426 13:12:05.999847   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:06.002071  1234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:06.003037  1235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:06.003952  1237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:06.004791   398 server_base.cc:1034] running on GCE node
I20240426 13:12:06.004896   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:06.004925   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:06.004937   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137126004937 us; error 0 us; skew 500 ppm
I20240426 13:12:06.005723   398 webserver.cc:460] Webserver started at http://127.0.99.190:38983/ using document root <none> and password file <none>
I20240426 13:12:06.005833   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:06.005867   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:06.005918   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:06.006280   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "d82e8f7b8f284e8089d0e6fb2b58f720"
format_stamp: "Formatted at 2024-04-26 13:12:06 on dist-test-slave-2jsl"
I20240426 13:12:06.007534   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:06.008402  1242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.008699   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:06.008755   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "d82e8f7b8f284e8089d0e6fb2b58f720"
format_stamp: "Formatted at 2024-04-26 13:12:06 on dist-test-slave-2jsl"
I20240426 13:12:06.008822   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:06.012349   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:06.012574   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:06.016031   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:38713
I20240426 13:12:06.017115  1295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:06.021054  1294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:38713 every 8 connection(s)
I20240426 13:12:06.022033  1295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720: Bootstrap starting.
I20240426 13:12:06.022611  1295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:06.023381  1295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720: No bootstrap required, opened a new log
I20240426 13:12:06.023674  1295 raft_consensus.cc:348] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82e8f7b8f284e8089d0e6fb2b58f720" member_type: VOTER }
I20240426 13:12:06.023744  1295 raft_consensus.cc:374] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:06.023777  1295 raft_consensus.cc:729] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d82e8f7b8f284e8089d0e6fb2b58f720, State: Initialized, Role: FOLLOWER
I20240426 13:12:06.023892  1295 consensus_queue.cc:260] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82e8f7b8f284e8089d0e6fb2b58f720" member_type: VOTER }
I20240426 13:12:06.023950  1295 raft_consensus.cc:388] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:12:06.023986  1295 raft_consensus.cc:482] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:12:06.024034  1295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:06.024616  1295 raft_consensus.cc:504] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82e8f7b8f284e8089d0e6fb2b58f720" member_type: VOTER }
I20240426 13:12:06.024706  1295 leader_election.cc:302] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d82e8f7b8f284e8089d0e6fb2b58f720; no voters: 
I20240426 13:12:06.024856  1295 leader_election.cc:288] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:12:06.024933  1298 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:06.025094  1298 raft_consensus.cc:686] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 1 LEADER]: Becoming Leader. State: Replica: d82e8f7b8f284e8089d0e6fb2b58f720, State: Running, Role: LEADER
I20240426 13:12:06.025149  1295 sys_catalog.cc:564] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:12:06.025200  1298 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82e8f7b8f284e8089d0e6fb2b58f720" member_type: VOTER }
I20240426 13:12:06.025580  1299 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d82e8f7b8f284e8089d0e6fb2b58f720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82e8f7b8f284e8089d0e6fb2b58f720" member_type: VOTER } }
I20240426 13:12:06.025615  1300 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d82e8f7b8f284e8089d0e6fb2b58f720. Latest consensus state: current_term: 1 leader_uuid: "d82e8f7b8f284e8089d0e6fb2b58f720" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82e8f7b8f284e8089d0e6fb2b58f720" member_type: VOTER } }
I20240426 13:12:06.025734  1300 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:06.025971  1299 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:06.026176  1306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:12:06.026834  1306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:12:06.026947   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:12:06.028156  1306 catalog_manager.cc:1347] Generated new cluster ID: e8eb40e69c2d4b1681e71e64921362ae
I20240426 13:12:06.028198  1306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:12:06.046066  1306 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:12:06.046448  1306 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:12:06.052654  1306 catalog_manager.cc:5857] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720: Generated new TSK 0
I20240426 13:12:06.052759  1306 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:12:06.058841   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:06.060935  1316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:06.061733  1317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:06.062459  1319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:06.063150   398 server_base.cc:1034] running on GCE node
I20240426 13:12:06.063253   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:06.063285   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:06.063298   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137126063298 us; error 0 us; skew 500 ppm
I20240426 13:12:06.064090   398 webserver.cc:460] Webserver started at http://127.0.99.129:32851/ using document root <none> and password file <none>
I20240426 13:12:06.064195   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:06.064224   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:06.064265   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:06.064576   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "69fc7e8f37a84f429093796903d92854"
format_stamp: "Formatted at 2024-04-26 13:12:06 on dist-test-slave-2jsl"
I20240426 13:12:06.065698   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:06.066422  1324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.066610   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.066665   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "69fc7e8f37a84f429093796903d92854"
format_stamp: "Formatted at 2024-04-26 13:12:06 on dist-test-slave-2jsl"
I20240426 13:12:06.066721   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:06.074657   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:06.074906   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:06.075124   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:06.075495   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:06.075529   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.075567   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:06.075601   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.079039   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:35987
I20240426 13:12:06.079067  1387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:35987 every 8 connection(s)
I20240426 13:12:06.079639   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:06.081454  1392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:06.082141  1393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:06.082850  1395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:06.083425  1388 heartbeater.cc:347] Connected to a master server at 127.0.99.190:38713
I20240426 13:12:06.083539  1388 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:06.083665   398 server_base.cc:1034] running on GCE node
I20240426 13:12:06.083752  1388 heartbeater.cc:511] Master 127.0.99.190:38713 requested a full tablet report, sending...
I20240426 13:12:06.083848   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:06.083894   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:06.083916   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137126083915 us; error 0 us; skew 500 ppm
I20240426 13:12:06.084358  1259 ts_manager.cc:194] Registered new tserver with Master: 69fc7e8f37a84f429093796903d92854 (127.0.99.129:35987)
I20240426 13:12:06.084793   398 webserver.cc:460] Webserver started at http://127.0.99.130:37419/ using document root <none> and password file <none>
I20240426 13:12:06.084934   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:06.084983   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:06.085045   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:06.085314  1259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40354
I20240426 13:12:06.085399   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "bb6181ecae1b41bab32fbab1f467e75b"
format_stamp: "Formatted at 2024-04-26 13:12:06 on dist-test-slave-2jsl"
I20240426 13:12:06.086728   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:12:06.087431  1400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.087615   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:12:06.087677   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "bb6181ecae1b41bab32fbab1f467e75b"
format_stamp: "Formatted at 2024-04-26 13:12:06 on dist-test-slave-2jsl"
I20240426 13:12:06.087736   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:06.104298   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:06.104635   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:06.104864   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:06.105283   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:06.105324   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.105381   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:06.105414   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.109370   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:38861
I20240426 13:12:06.109390  1463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:38861 every 8 connection(s)
I20240426 13:12:06.110059   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:06.112371  1468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:06.113065  1467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:06.113829  1470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:06.113898  1464 heartbeater.cc:347] Connected to a master server at 127.0.99.190:38713
I20240426 13:12:06.114017  1464 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:06.114200  1464 heartbeater.cc:511] Master 127.0.99.190:38713 requested a full tablet report, sending...
I20240426 13:12:06.114636   398 server_base.cc:1034] running on GCE node
I20240426 13:12:06.114813   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20240426 13:12:06.114856  1259 ts_manager.cc:194] Registered new tserver with Master: bb6181ecae1b41bab32fbab1f467e75b (127.0.99.130:38861)
W20240426 13:12:06.114861   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:06.114928   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137126114927 us; error 0 us; skew 500 ppm
I20240426 13:12:06.115782   398 webserver.cc:460] Webserver started at http://127.0.99.131:33443/ using document root <none> and password file <none>
I20240426 13:12:06.115911   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:06.115939  1259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40358
I20240426 13:12:06.115960   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:06.116062   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:06.116461   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "a46a5e28a7064457afece2aaebac4847"
format_stamp: "Formatted at 2024-04-26 13:12:06 on dist-test-slave-2jsl"
I20240426 13:12:06.117707   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:12:06.118463  1475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.118671   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.118747   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "a46a5e28a7064457afece2aaebac4847"
format_stamp: "Formatted at 2024-04-26 13:12:06 on dist-test-slave-2jsl"
I20240426 13:12:06.118811   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestWaitOnAbortCommit.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:06.123207   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:06.123577   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:06.123843   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:06.124234   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:06.124280   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.124328   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:06.124364   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:06.128824   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:42439
I20240426 13:12:06.128842  1538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:42439 every 8 connection(s)
I20240426 13:12:06.132445  1539 heartbeater.cc:347] Connected to a master server at 127.0.99.190:38713
I20240426 13:12:06.132506  1539 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:06.132624  1539 heartbeater.cc:511] Master 127.0.99.190:38713 requested a full tablet report, sending...
I20240426 13:12:06.133071  1259 ts_manager.cc:194] Registered new tserver with Master: a46a5e28a7064457afece2aaebac4847 (127.0.99.131:42439)
I20240426 13:12:06.133617  1259 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40362
I20240426 13:12:06.133617   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00452099s
I20240426 13:12:06.134202   398 test_util.cc:268] Using random seed: -1501568585
I20240426 13:12:06.140406  1259 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40364:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:12:06.140714  1259 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:12:06.148577  1352 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8360698b7342dca406f0481a34c500 (DEFAULT_TABLE table=test-workload [id=4caca6941bbf4bc2b0637ad94fe044c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:06.148834  1352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8360698b7342dca406f0481a34c500. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:06.148871  1503 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8360698b7342dca406f0481a34c500 (DEFAULT_TABLE table=test-workload [id=4caca6941bbf4bc2b0637ad94fe044c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:06.149081  1503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8360698b7342dca406f0481a34c500. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:06.150699  1555 tablet_bootstrap.cc:492] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847: Bootstrap starting.
I20240426 13:12:06.150904  1554 tablet_bootstrap.cc:492] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854: Bootstrap starting.
I20240426 13:12:06.151470  1555 tablet_bootstrap.cc:654] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:06.151612  1428 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8360698b7342dca406f0481a34c500 (DEFAULT_TABLE table=test-workload [id=4caca6941bbf4bc2b0637ad94fe044c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:06.151758  1554 tablet_bootstrap.cc:654] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:06.151813  1428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8360698b7342dca406f0481a34c500. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:06.152387  1555 tablet_bootstrap.cc:492] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847: No bootstrap required, opened a new log
I20240426 13:12:06.152468  1555 ts_tablet_manager.cc:1367] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:06.152695  1554 tablet_bootstrap.cc:492] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854: No bootstrap required, opened a new log
I20240426 13:12:06.152773  1554 ts_tablet_manager.cc:1367] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:06.152842  1555 raft_consensus.cc:348] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.152931  1555 raft_consensus.cc:374] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:06.152962  1555 raft_consensus.cc:729] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a46a5e28a7064457afece2aaebac4847, State: Initialized, Role: FOLLOWER
I20240426 13:12:06.153059  1555 consensus_queue.cc:260] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.153146  1554 raft_consensus.cc:348] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.153246  1554 raft_consensus.cc:374] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:06.153295  1554 raft_consensus.cc:729] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69fc7e8f37a84f429093796903d92854, State: Initialized, Role: FOLLOWER
I20240426 13:12:06.153391  1555 ts_tablet_manager.cc:1398] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:06.153553  1539 heartbeater.cc:503] Master 127.0.99.190:38713 was elected leader, sending a full tablet report...
I20240426 13:12:06.153486  1554 consensus_queue.cc:260] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.153867  1554 ts_tablet_manager.cc:1398] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:06.154047  1388 heartbeater.cc:503] Master 127.0.99.190:38713 was elected leader, sending a full tablet report...
I20240426 13:12:06.154247  1559 tablet_bootstrap.cc:492] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b: Bootstrap starting.
I20240426 13:12:06.155069  1559 tablet_bootstrap.cc:654] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:06.155984  1559 tablet_bootstrap.cc:492] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b: No bootstrap required, opened a new log
I20240426 13:12:06.156071  1559 ts_tablet_manager.cc:1367] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:06.156409  1559 raft_consensus.cc:348] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.156481  1559 raft_consensus.cc:374] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:06.156499  1559 raft_consensus.cc:729] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb6181ecae1b41bab32fbab1f467e75b, State: Initialized, Role: FOLLOWER
I20240426 13:12:06.156569  1559 consensus_queue.cc:260] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.156798  1559 ts_tablet_manager.cc:1398] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:06.156881  1464 heartbeater.cc:503] Master 127.0.99.190:38713 was elected leader, sending a full tablet report...
I20240426 13:12:06.217334  1558 raft_consensus.cc:482] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:12:06.217414  1558 raft_consensus.cc:504] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.217804  1558 leader_election.cc:288] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb6181ecae1b41bab32fbab1f467e75b (127.0.99.130:38861), 69fc7e8f37a84f429093796903d92854 (127.0.99.129:35987)
I20240426 13:12:06.221114  1362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8360698b7342dca406f0481a34c500" candidate_uuid: "a46a5e28a7064457afece2aaebac4847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69fc7e8f37a84f429093796903d92854" is_pre_election: true
I20240426 13:12:06.221230  1362 raft_consensus.cc:2454] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a46a5e28a7064457afece2aaebac4847 in term 0.
I20240426 13:12:06.221330  1438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8360698b7342dca406f0481a34c500" candidate_uuid: "a46a5e28a7064457afece2aaebac4847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb6181ecae1b41bab32fbab1f467e75b" is_pre_election: true
I20240426 13:12:06.221441  1438 raft_consensus.cc:2454] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a46a5e28a7064457afece2aaebac4847 in term 0.
I20240426 13:12:06.221529  1477 leader_election.cc:302] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69fc7e8f37a84f429093796903d92854, a46a5e28a7064457afece2aaebac4847; no voters: 
I20240426 13:12:06.221681  1558 raft_consensus.cc:2781] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:12:06.221743  1558 raft_consensus.cc:482] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:12:06.221768  1558 raft_consensus.cc:3037] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:06.222455  1558 raft_consensus.cc:504] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.222710  1558 leader_election.cc:288] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [CANDIDATE]: Term 1 election: Requested vote from peers bb6181ecae1b41bab32fbab1f467e75b (127.0.99.130:38861), 69fc7e8f37a84f429093796903d92854 (127.0.99.129:35987)
I20240426 13:12:06.222986  1438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8360698b7342dca406f0481a34c500" candidate_uuid: "a46a5e28a7064457afece2aaebac4847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb6181ecae1b41bab32fbab1f467e75b"
I20240426 13:12:06.222986  1362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8360698b7342dca406f0481a34c500" candidate_uuid: "a46a5e28a7064457afece2aaebac4847" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69fc7e8f37a84f429093796903d92854"
I20240426 13:12:06.223117  1438 raft_consensus.cc:3037] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:06.223153  1362 raft_consensus.cc:3037] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:06.223899  1362 raft_consensus.cc:2454] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a46a5e28a7064457afece2aaebac4847 in term 1.
I20240426 13:12:06.224020  1438 raft_consensus.cc:2454] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a46a5e28a7064457afece2aaebac4847 in term 1.
I20240426 13:12:06.224157  1477 leader_election.cc:302] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69fc7e8f37a84f429093796903d92854, a46a5e28a7064457afece2aaebac4847; no voters: 
I20240426 13:12:06.224340  1558 raft_consensus.cc:2781] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:06.224509  1558 raft_consensus.cc:686] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 1 LEADER]: Becoming Leader. State: Replica: a46a5e28a7064457afece2aaebac4847, State: Running, Role: LEADER
I20240426 13:12:06.224675  1558 consensus_queue.cc:237] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:06.226001  1259 catalog_manager.cc:5484] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 reported cstate change: term changed from 0 to 1, leader changed from <none> to a46a5e28a7064457afece2aaebac4847 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "a46a5e28a7064457afece2aaebac4847" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } health_report { overall_health: UNKNOWN } } }
I20240426 13:12:06.259521  1438 raft_consensus.cc:1261] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 1 FOLLOWER]: Refusing update from remote peer a46a5e28a7064457afece2aaebac4847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:06.259804  1362 raft_consensus.cc:1261] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 1 FOLLOWER]: Refusing update from remote peer a46a5e28a7064457afece2aaebac4847: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:06.259869  1564 consensus_queue.cc:1035] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:06.260262  1564 consensus_queue.cc:1035] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:16.787470  1608 raft_consensus.cc:979] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847: : Instructing follower 69fc7e8f37a84f429093796903d92854 to start an election
I20240426 13:12:16.787552  1608 raft_consensus.cc:1067] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 1 LEADER]: Signalling peer 69fc7e8f37a84f429093796903d92854 to start an election
I20240426 13:12:16.787896  1361 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5e8360698b7342dca406f0481a34c500"
dest_uuid: "69fc7e8f37a84f429093796903d92854"
 from {username='slave'} at 127.0.0.1:34512
I20240426 13:12:16.788019  1361 raft_consensus.cc:482] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:12:16.788074  1361 raft_consensus.cc:3037] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:16.788902  1361 raft_consensus.cc:504] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:16.789301  1361 leader_election.cc:288] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [CANDIDATE]: Term 2 election: Requested vote from peers a46a5e28a7064457afece2aaebac4847 (127.0.99.131:42439), bb6181ecae1b41bab32fbab1f467e75b (127.0.99.130:38861)
I20240426 13:12:16.792726  1513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8360698b7342dca406f0481a34c500" candidate_uuid: "69fc7e8f37a84f429093796903d92854" candidate_term: 2 candidate_status { last_received { term: 1 index: 3053 } } ignore_live_leader: true dest_uuid: "a46a5e28a7064457afece2aaebac4847"
I20240426 13:12:16.792858  1513 raft_consensus.cc:3032] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:12:16.792902  1513 raft_consensus.cc:729] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a46a5e28a7064457afece2aaebac4847, State: Running, Role: LEADER
I20240426 13:12:16.792985  1438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8360698b7342dca406f0481a34c500" candidate_uuid: "69fc7e8f37a84f429093796903d92854" candidate_term: 2 candidate_status { last_received { term: 1 index: 3053 } } ignore_live_leader: true dest_uuid: "bb6181ecae1b41bab32fbab1f467e75b"
I20240426 13:12:16.793038  1513 consensus_queue.cc:260] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3053, Committed index: 3053, Last appended: 1.3053, Last appended by leader: 3053, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:16.793112  1438 raft_consensus.cc:3037] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:16.793179  1513 raft_consensus.cc:3037] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:16.793845  1438 raft_consensus.cc:2454] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69fc7e8f37a84f429093796903d92854 in term 2.
I20240426 13:12:16.793917  1513 raft_consensus.cc:2454] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69fc7e8f37a84f429093796903d92854 in term 2.
I20240426 13:12:16.794139  1328 leader_election.cc:302] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69fc7e8f37a84f429093796903d92854, bb6181ecae1b41bab32fbab1f467e75b; no voters: 
I20240426 13:12:16.794456  1614 raft_consensus.cc:2781] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:12:16.794617  1614 raft_consensus.cc:686] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 2 LEADER]: Becoming Leader. State: Replica: 69fc7e8f37a84f429093796903d92854, State: Running, Role: LEADER
I20240426 13:12:16.794751  1614 consensus_queue.cc:237] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3053, Committed index: 3053, Last appended: 1.3053, Last appended by leader: 3053, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } }
I20240426 13:12:16.795992  1254 catalog_manager.cc:5484] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 reported cstate change: term changed from 1 to 2, leader changed from a46a5e28a7064457afece2aaebac4847 (127.0.99.131) to 69fc7e8f37a84f429093796903d92854 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "69fc7e8f37a84f429093796903d92854" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69fc7e8f37a84f429093796903d92854" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35987 } health_report { overall_health: HEALTHY } } }
I20240426 13:12:16.800092  1513 raft_consensus.cc:1261] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 2 FOLLOWER]: Refusing update from remote peer 69fc7e8f37a84f429093796903d92854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3053. Preceding OpId from leader: term: 2 index: 3055. (index mismatch)
I20240426 13:12:16.800117  1438 raft_consensus.cc:1261] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 2 FOLLOWER]: Refusing update from remote peer 69fc7e8f37a84f429093796903d92854: Log matching property violated. Preceding OpId in replica: term: 1 index: 3053. Preceding OpId from leader: term: 2 index: 3055. (index mismatch)
I20240426 13:12:16.800478  1615 consensus_queue.cc:1035] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a46a5e28a7064457afece2aaebac4847" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3054, Last known committed idx: 3053, Time since last communication: 0.000s
I20240426 13:12:16.800626  1614 consensus_queue.cc:1035] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb6181ecae1b41bab32fbab1f467e75b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3054, Last known committed idx: 3053, Time since last communication: 0.000s
I20240426 13:12:16.802537  1621 mvcc.cc:205] Tried to move back new op lower bound from 7021105712330838016 to 7021105712312594432. Current Snapshot: MvccSnapshot[applied={T|T < 7021105712330838016}]
I20240426 13:12:17.841531   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:12:17.845086   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:17.845245   398 tablet_replica.cc:330] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854: stopping tablet replica
I20240426 13:12:17.845360   398 raft_consensus.cc:2229] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:12:17.845528   398 raft_consensus.cc:2258] T 5e8360698b7342dca406f0481a34c500 P 69fc7e8f37a84f429093796903d92854 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:17.860836   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:12:17.862412   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:12:17.866488   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:17.866627   398 tablet_replica.cc:330] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b: stopping tablet replica
I20240426 13:12:17.866729   398 raft_consensus.cc:2229] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:17.866822   398 raft_consensus.cc:2258] T 5e8360698b7342dca406f0481a34c500 P bb6181ecae1b41bab32fbab1f467e75b [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:17.881857   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:12:17.883301   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:12:17.887082   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:17.887235   398 tablet_replica.cc:330] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847: stopping tablet replica
I20240426 13:12:17.887334   398 raft_consensus.cc:2229] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:17.887431   398 raft_consensus.cc:2258] T 5e8360698b7342dca406f0481a34c500 P a46a5e28a7064457afece2aaebac4847 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:17.902577   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:12:17.904160   398 master.cc:537] Master@127.0.99.190:38713 shutting down...
I20240426 13:12:17.907260   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:17.907572   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:17.907644   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P d82e8f7b8f284e8089d0e6fb2b58f720: stopping tablet replica
I20240426 13:12:17.920236   398 master.cc:552] Master@127.0.99.190:38713 shutdown complete.
[       OK ] TxnParticipantITest.TestWaitOnAbortCommit (11928 ms)
[ RUN      ] TxnParticipantITest.TestProxyBasicCalls
I20240426 13:12:17.928236   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:40553
I20240426 13:12:17.928514   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:17.931064  1633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:17.931923  1634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:17.932705  1636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:17.933578   398 server_base.cc:1034] running on GCE node
I20240426 13:12:17.933710   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:17.933743   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:17.933756   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137137933756 us; error 0 us; skew 500 ppm
I20240426 13:12:17.938949   398 webserver.cc:460] Webserver started at http://127.0.99.190:41779/ using document root <none> and password file <none>
I20240426 13:12:17.939060   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:17.939095   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:17.939141   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:17.939471   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "31bd124bc9444d569eecb6474541402d"
format_stamp: "Formatted at 2024-04-26 13:12:17 on dist-test-slave-2jsl"
I20240426 13:12:17.940796   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:17.941540  1641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:17.941766   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:17.941823   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "31bd124bc9444d569eecb6474541402d"
format_stamp: "Formatted at 2024-04-26 13:12:17 on dist-test-slave-2jsl"
I20240426 13:12:17.941888   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:17.948019   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:17.948258   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:17.951808   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:40553
I20240426 13:12:17.954924  1694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:17.957042  1693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:40553 every 8 connection(s)
I20240426 13:12:17.957788  1694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d: Bootstrap starting.
I20240426 13:12:17.958410  1694 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:17.959127  1694 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d: No bootstrap required, opened a new log
I20240426 13:12:17.959398  1694 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bd124bc9444d569eecb6474541402d" member_type: VOTER }
I20240426 13:12:17.959463  1694 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:17.959487  1694 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31bd124bc9444d569eecb6474541402d, State: Initialized, Role: FOLLOWER
I20240426 13:12:17.959565  1694 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bd124bc9444d569eecb6474541402d" member_type: VOTER }
I20240426 13:12:17.959609  1694 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:12:17.959633  1694 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:12:17.959663  1694 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:17.960145  1694 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bd124bc9444d569eecb6474541402d" member_type: VOTER }
I20240426 13:12:17.960237  1694 leader_election.cc:302] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31bd124bc9444d569eecb6474541402d; no voters: 
I20240426 13:12:17.960352  1694 leader_election.cc:288] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:12:17.960451  1697 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:17.960603  1697 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 1 LEADER]: Becoming Leader. State: Replica: 31bd124bc9444d569eecb6474541402d, State: Running, Role: LEADER
I20240426 13:12:17.960708  1694 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:12:17.960722  1697 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bd124bc9444d569eecb6474541402d" member_type: VOTER }
I20240426 13:12:17.961127  1698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31bd124bc9444d569eecb6474541402d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bd124bc9444d569eecb6474541402d" member_type: VOTER } }
I20240426 13:12:17.961172  1699 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31bd124bc9444d569eecb6474541402d. Latest consensus state: current_term: 1 leader_uuid: "31bd124bc9444d569eecb6474541402d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31bd124bc9444d569eecb6474541402d" member_type: VOTER } }
I20240426 13:12:17.961302  1698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:17.961319  1699 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:17.961756  1704 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:12:17.962376  1704 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:12:17.962564   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:12:17.963874  1704 catalog_manager.cc:1347] Generated new cluster ID: 67f3908deac645c8a1b40f350737c1d5
I20240426 13:12:17.963940  1704 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:12:17.971756  1704 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:12:17.972155  1704 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:12:17.976773  1704 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d: Generated new TSK 0
I20240426 13:12:17.976894  1704 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:12:17.978348   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:17.980291  1715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:17.981062  1716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:17.981777  1718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:17.982542   398 server_base.cc:1034] running on GCE node
I20240426 13:12:17.982656   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:17.982688   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:17.982709   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137137982709 us; error 0 us; skew 500 ppm
I20240426 13:12:17.983397   398 webserver.cc:460] Webserver started at http://127.0.99.129:42605/ using document root <none> and password file <none>
I20240426 13:12:17.983510   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:17.983548   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:17.983611   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:17.983886   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "92d80f653b5242ccb759356cf486a271"
format_stamp: "Formatted at 2024-04-26 13:12:17 on dist-test-slave-2jsl"
I20240426 13:12:17.985132   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:17.986002  1723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:17.986191   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:17.986249   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "92d80f653b5242ccb759356cf486a271"
format_stamp: "Formatted at 2024-04-26 13:12:17 on dist-test-slave-2jsl"
I20240426 13:12:17.986307   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:17.992985   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:17.993204   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:17.993399   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:17.993754   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:17.993788   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:17.993830   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:17.993857   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:17.997362   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:42263
I20240426 13:12:17.997383  1786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:42263 every 8 connection(s)
I20240426 13:12:17.998035   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:18.000099  1791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:18.000798  1794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:18.001520   398 server_base.cc:1034] running on GCE node
I20240426 13:12:18.001832  1787 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40553
I20240426 13:12:18.001897  1787 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:18.002068  1787 heartbeater.cc:511] Master 127.0.99.190:40553 requested a full tablet report, sending...
W20240426 13:12:18.002243  1792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:18.002491   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:18.002545   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:18.002565   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137138002565 us; error 0 us; skew 500 ppm
I20240426 13:12:18.002696  1658 ts_manager.cc:194] Registered new tserver with Master: 92d80f653b5242ccb759356cf486a271 (127.0.99.129:42263)
I20240426 13:12:18.003281   398 webserver.cc:460] Webserver started at http://127.0.99.130:35717/ using document root <none> and password file <none>
I20240426 13:12:18.003402   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:18.003441   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:18.003499   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:18.003721  1658 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42444
I20240426 13:12:18.003787   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "f4ca57df788146a1acf5f46eca5ada39"
format_stamp: "Formatted at 2024-04-26 13:12:18 on dist-test-slave-2jsl"
I20240426 13:12:18.005187   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:12:18.006457  1799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:18.006645   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:18.006703   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "f4ca57df788146a1acf5f46eca5ada39"
format_stamp: "Formatted at 2024-04-26 13:12:18 on dist-test-slave-2jsl"
I20240426 13:12:18.006764   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:18.021181   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:18.021407   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:18.021612   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:18.021917   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:18.021950   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:18.022032   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:18.022059   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:18.025516   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:37055
I20240426 13:12:18.025568  1862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:37055 every 8 connection(s)
I20240426 13:12:18.026183   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:18.028331  1866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:18.029134  1867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:18.029886  1863 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40553
I20240426 13:12:18.029958  1863 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:18.029960   398 server_base.cc:1034] running on GCE node
I20240426 13:12:18.030146  1863 heartbeater.cc:511] Master 127.0.99.190:40553 requested a full tablet report, sending...
W20240426 13:12:18.030593  1869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:18.030671  1658 ts_manager.cc:194] Registered new tserver with Master: f4ca57df788146a1acf5f46eca5ada39 (127.0.99.130:37055)
I20240426 13:12:18.030790   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:18.030831   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:18.030858   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137138030857 us; error 0 us; skew 500 ppm
I20240426 13:12:18.031235  1658 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42448
I20240426 13:12:18.031611   398 webserver.cc:460] Webserver started at http://127.0.99.131:36475/ using document root <none> and password file <none>
I20240426 13:12:18.031741   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:18.031782   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:18.031841   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:18.032105   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "a492b890d5f04c80bf07adbf8b2d9348"
format_stamp: "Formatted at 2024-04-26 13:12:18 on dist-test-slave-2jsl"
I20240426 13:12:18.033285   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:12:18.034024  1874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:18.034229   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:18.034284   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "a492b890d5f04c80bf07adbf8b2d9348"
format_stamp: "Formatted at 2024-04-26 13:12:18 on dist-test-slave-2jsl"
I20240426 13:12:18.034330   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyBasicCalls.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:18.051395   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:18.051590   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:18.051761   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:18.052078   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:18.052129   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:18.052166   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:18.052196   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:18.056082   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:38733
I20240426 13:12:18.056125  1937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:38733 every 8 connection(s)
I20240426 13:12:18.059448  1938 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40553
I20240426 13:12:18.059511  1938 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:18.059671  1938 heartbeater.cc:511] Master 127.0.99.190:40553 requested a full tablet report, sending...
I20240426 13:12:18.060109  1658 ts_manager.cc:194] Registered new tserver with Master: a492b890d5f04c80bf07adbf8b2d9348 (127.0.99.131:38733)
I20240426 13:12:18.060531  1658 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42452
I20240426 13:12:18.060822   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004407723s
I20240426 13:12:18.061262   398 test_util.cc:268] Using random seed: -1489641522
I20240426 13:12:18.067095  1658 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42466:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:12:18.067413  1658 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:12:18.075276  1902 tablet_service.cc:1467] Processing CreateTablet for tablet f10c6ed34f3449d2ae18836548c8e371 (DEFAULT_TABLE table=test-workload [id=33318452122c4c70ad42075cc96bef91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:18.075338  1827 tablet_service.cc:1467] Processing CreateTablet for tablet f10c6ed34f3449d2ae18836548c8e371 (DEFAULT_TABLE table=test-workload [id=33318452122c4c70ad42075cc96bef91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:18.075536  1902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10c6ed34f3449d2ae18836548c8e371. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:18.075595  1827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10c6ed34f3449d2ae18836548c8e371. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:18.075708  1751 tablet_service.cc:1467] Processing CreateTablet for tablet f10c6ed34f3449d2ae18836548c8e371 (DEFAULT_TABLE table=test-workload [id=33318452122c4c70ad42075cc96bef91]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:18.075923  1751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f10c6ed34f3449d2ae18836548c8e371. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:18.077409  1953 tablet_bootstrap.cc:492] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39: Bootstrap starting.
I20240426 13:12:18.077770  1955 tablet_bootstrap.cc:492] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271: Bootstrap starting.
I20240426 13:12:18.078284  1953 tablet_bootstrap.cc:654] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:18.078594  1955 tablet_bootstrap.cc:654] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:18.079145  1954 tablet_bootstrap.cc:492] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348: Bootstrap starting.
I20240426 13:12:18.079228  1953 tablet_bootstrap.cc:492] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39: No bootstrap required, opened a new log
I20240426 13:12:18.079294  1953 ts_tablet_manager.cc:1367] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:18.079443  1955 tablet_bootstrap.cc:492] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271: No bootstrap required, opened a new log
I20240426 13:12:18.079514  1955 ts_tablet_manager.cc:1367] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:18.079663  1953 raft_consensus.cc:348] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.079762  1953 raft_consensus.cc:374] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:18.079800  1953 raft_consensus.cc:729] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4ca57df788146a1acf5f46eca5ada39, State: Initialized, Role: FOLLOWER
I20240426 13:12:18.079847  1955 raft_consensus.cc:348] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.079943  1955 raft_consensus.cc:374] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:18.079993  1955 raft_consensus.cc:729] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92d80f653b5242ccb759356cf486a271, State: Initialized, Role: FOLLOWER
I20240426 13:12:18.079931  1953 consensus_queue.cc:260] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.080121  1955 consensus_queue.cc:260] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.080305  1953 ts_tablet_manager.cc:1398] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:18.080334  1863 heartbeater.cc:503] Master 127.0.99.190:40553 was elected leader, sending a full tablet report...
I20240426 13:12:18.079938  1954 tablet_bootstrap.cc:654] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:18.080420  1955 ts_tablet_manager.cc:1398] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:18.080570  1787 heartbeater.cc:503] Master 127.0.99.190:40553 was elected leader, sending a full tablet report...
I20240426 13:12:18.081615  1954 tablet_bootstrap.cc:492] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348: No bootstrap required, opened a new log
I20240426 13:12:18.081698  1954 ts_tablet_manager.cc:1367] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:12:18.082063  1954 raft_consensus.cc:348] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.082165  1954 raft_consensus.cc:374] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:18.082203  1954 raft_consensus.cc:729] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a492b890d5f04c80bf07adbf8b2d9348, State: Initialized, Role: FOLLOWER
I20240426 13:12:18.082309  1954 consensus_queue.cc:260] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.082641  1954 ts_tablet_manager.cc:1398] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:18.082662  1938 heartbeater.cc:503] Master 127.0.99.190:40553 was elected leader, sending a full tablet report...
I20240426 13:12:18.094121  1958 raft_consensus.cc:482] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:12:18.094192  1958 raft_consensus.cc:504] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.094585  1958 leader_election.cc:288] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92d80f653b5242ccb759356cf486a271 (127.0.99.129:42263), a492b890d5f04c80bf07adbf8b2d9348 (127.0.99.131:38733)
I20240426 13:12:18.097575  1761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f10c6ed34f3449d2ae18836548c8e371" candidate_uuid: "f4ca57df788146a1acf5f46eca5ada39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d80f653b5242ccb759356cf486a271" is_pre_election: true
I20240426 13:12:18.097666  1761 raft_consensus.cc:2454] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4ca57df788146a1acf5f46eca5ada39 in term 0.
I20240426 13:12:18.097910  1912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f10c6ed34f3449d2ae18836548c8e371" candidate_uuid: "f4ca57df788146a1acf5f46eca5ada39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a492b890d5f04c80bf07adbf8b2d9348" is_pre_election: true
I20240426 13:12:18.097923  1802 leader_election.cc:302] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d80f653b5242ccb759356cf486a271, f4ca57df788146a1acf5f46eca5ada39; no voters: 
I20240426 13:12:18.098043  1912 raft_consensus.cc:2454] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4ca57df788146a1acf5f46eca5ada39 in term 0.
I20240426 13:12:18.098095  1958 raft_consensus.cc:2781] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:12:18.098156  1958 raft_consensus.cc:482] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:12:18.098205  1958 raft_consensus.cc:3037] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:18.098826  1958 raft_consensus.cc:504] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.099089  1958 leader_election.cc:288] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [CANDIDATE]: Term 1 election: Requested vote from peers 92d80f653b5242ccb759356cf486a271 (127.0.99.129:42263), a492b890d5f04c80bf07adbf8b2d9348 (127.0.99.131:38733)
I20240426 13:12:18.099299  1761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f10c6ed34f3449d2ae18836548c8e371" candidate_uuid: "f4ca57df788146a1acf5f46eca5ada39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92d80f653b5242ccb759356cf486a271"
I20240426 13:12:18.099328  1912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f10c6ed34f3449d2ae18836548c8e371" candidate_uuid: "f4ca57df788146a1acf5f46eca5ada39" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a492b890d5f04c80bf07adbf8b2d9348"
I20240426 13:12:18.099383  1761 raft_consensus.cc:3037] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:18.099428  1912 raft_consensus.cc:3037] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:18.100050  1761 raft_consensus.cc:2454] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4ca57df788146a1acf5f46eca5ada39 in term 1.
I20240426 13:12:18.100131  1912 raft_consensus.cc:2454] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4ca57df788146a1acf5f46eca5ada39 in term 1.
I20240426 13:12:18.100276  1802 leader_election.cc:302] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d80f653b5242ccb759356cf486a271, f4ca57df788146a1acf5f46eca5ada39; no voters: 
I20240426 13:12:18.100454  1958 raft_consensus.cc:2781] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:18.100620  1958 raft_consensus.cc:686] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 1 LEADER]: Becoming Leader. State: Replica: f4ca57df788146a1acf5f46eca5ada39, State: Running, Role: LEADER
I20240426 13:12:18.100728  1958 consensus_queue.cc:237] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:18.101821  1658 catalog_manager.cc:5484] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4ca57df788146a1acf5f46eca5ada39 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "f4ca57df788146a1acf5f46eca5ada39" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } health_report { overall_health: UNKNOWN } } }
I20240426 13:12:18.131506  1761 raft_consensus.cc:1261] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 1 FOLLOWER]: Refusing update from remote peer f4ca57df788146a1acf5f46eca5ada39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:18.131561  1912 raft_consensus.cc:1261] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 1 FOLLOWER]: Refusing update from remote peer f4ca57df788146a1acf5f46eca5ada39: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:18.131844  1958 consensus_queue.cc:1035] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:18.132004  1963 consensus_queue.cc:1035] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:28.560237  1996 raft_consensus.cc:979] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39: : Instructing follower 92d80f653b5242ccb759356cf486a271 to start an election
I20240426 13:12:28.560354  1996 raft_consensus.cc:1067] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 1 LEADER]: Signalling peer 92d80f653b5242ccb759356cf486a271 to start an election
I20240426 13:12:28.560777  1760 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f10c6ed34f3449d2ae18836548c8e371"
dest_uuid: "92d80f653b5242ccb759356cf486a271"
 from {username='slave'} at 127.0.0.1:38428
I20240426 13:12:28.560899  1760 raft_consensus.cc:482] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:12:28.560966  1760 raft_consensus.cc:3037] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:28.561784  1760 raft_consensus.cc:504] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:28.562268  1760 leader_election.cc:288] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [CANDIDATE]: Term 2 election: Requested vote from peers f4ca57df788146a1acf5f46eca5ada39 (127.0.99.130:37055), a492b890d5f04c80bf07adbf8b2d9348 (127.0.99.131:38733)
I20240426 13:12:28.566499  1912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f10c6ed34f3449d2ae18836548c8e371" candidate_uuid: "92d80f653b5242ccb759356cf486a271" candidate_term: 2 candidate_status { last_received { term: 1 index: 3110 } } ignore_live_leader: true dest_uuid: "a492b890d5f04c80bf07adbf8b2d9348"
I20240426 13:12:28.566507  1837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f10c6ed34f3449d2ae18836548c8e371" candidate_uuid: "92d80f653b5242ccb759356cf486a271" candidate_term: 2 candidate_status { last_received { term: 1 index: 3110 } } ignore_live_leader: true dest_uuid: "f4ca57df788146a1acf5f46eca5ada39"
I20240426 13:12:28.566632  1837 raft_consensus.cc:3032] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:12:28.566632  1912 raft_consensus.cc:3037] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:28.566711  1837 raft_consensus.cc:729] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f4ca57df788146a1acf5f46eca5ada39, State: Running, Role: LEADER
I20240426 13:12:28.566887  1837 consensus_queue.cc:260] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3110, Committed index: 3110, Last appended: 1.3110, Last appended by leader: 3110, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:28.567037  1837 raft_consensus.cc:3037] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:12:28.567628  1912 raft_consensus.cc:2454] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d80f653b5242ccb759356cf486a271 in term 2.
I20240426 13:12:28.567909  1837 raft_consensus.cc:2454] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92d80f653b5242ccb759356cf486a271 in term 2.
I20240426 13:12:28.567941  1725 leader_election.cc:302] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92d80f653b5242ccb759356cf486a271, a492b890d5f04c80bf07adbf8b2d9348; no voters: 
I20240426 13:12:28.568210  2011 raft_consensus.cc:2781] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:12:28.568388  2011 raft_consensus.cc:686] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 2 LEADER]: Becoming Leader. State: Replica: 92d80f653b5242ccb759356cf486a271, State: Running, Role: LEADER
I20240426 13:12:28.568533  2011 consensus_queue.cc:237] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3110, Committed index: 3110, Last appended: 1.3110, Last appended by leader: 3110, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } }
I20240426 13:12:28.569885  1653 catalog_manager.cc:5484] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 reported cstate change: term changed from 1 to 2, leader changed from f4ca57df788146a1acf5f46eca5ada39 (127.0.99.130) to 92d80f653b5242ccb759356cf486a271 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "92d80f653b5242ccb759356cf486a271" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92d80f653b5242ccb759356cf486a271" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 } health_report { overall_health: UNKNOWN } } }
I20240426 13:12:28.575938  1912 raft_consensus.cc:1261] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 2 FOLLOWER]: Refusing update from remote peer 92d80f653b5242ccb759356cf486a271: Log matching property violated. Preceding OpId in replica: term: 1 index: 3110. Preceding OpId from leader: term: 2 index: 3112. (index mismatch)
I20240426 13:12:28.575996  1837 raft_consensus.cc:1261] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 2 FOLLOWER]: Refusing update from remote peer 92d80f653b5242ccb759356cf486a271: Log matching property violated. Preceding OpId in replica: term: 1 index: 3110. Preceding OpId from leader: term: 2 index: 3112. (index mismatch)
I20240426 13:12:28.576299  2012 consensus_queue.cc:1035] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a492b890d5f04c80bf07adbf8b2d9348" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3111, Last known committed idx: 3110, Time since last communication: 0.000s
I20240426 13:12:28.576416  2011 consensus_queue.cc:1035] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4ca57df788146a1acf5f46eca5ada39" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37055 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3111, Last known committed idx: 3110, Time since last communication: 0.000s
I20240426 13:12:28.584800   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:12:28.588312   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:28.588485   398 tablet_replica.cc:330] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271: stopping tablet replica
I20240426 13:12:28.588577   398 raft_consensus.cc:2229] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:12:28.588779   398 raft_consensus.cc:2258] T f10c6ed34f3449d2ae18836548c8e371 P 92d80f653b5242ccb759356cf486a271 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:28.605262   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:12:28.607177   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:12:28.610354   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:28.610589   398 tablet_replica.cc:330] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39: stopping tablet replica
I20240426 13:12:28.610692   398 raft_consensus.cc:2229] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:28.610819   398 raft_consensus.cc:2258] T f10c6ed34f3449d2ae18836548c8e371 P f4ca57df788146a1acf5f46eca5ada39 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:28.626890   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:12:28.628530   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:12:28.631707   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:28.631906   398 tablet_replica.cc:330] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348: stopping tablet replica
I20240426 13:12:28.632004   398 raft_consensus.cc:2229] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:28.632107   398 raft_consensus.cc:2258] T f10c6ed34f3449d2ae18836548c8e371 P a492b890d5f04c80bf07adbf8b2d9348 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:28.648103   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:12:28.649655   398 master.cc:537] Master@127.0.99.190:40553 shutting down...
I20240426 13:12:28.652771   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:28.652915   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:28.652978   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 31bd124bc9444d569eecb6474541402d: stopping tablet replica
I20240426 13:12:28.665127   398 master.cc:552] Master@127.0.99.190:40553 shutdown complete.
[       OK ] TxnParticipantITest.TestProxyBasicCalls (10745 ms)
[ RUN      ] TxnParticipantITest.TestBeginCommitAfterFinalize
I20240426 13:12:28.674046   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:43483
I20240426 13:12:28.674322   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:28.677240  2021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:28.678189  2024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:28.678853  2022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:28.679749   398 server_base.cc:1034] running on GCE node
I20240426 13:12:28.679878   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:28.679910   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:28.679929   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137148679928 us; error 0 us; skew 500 ppm
I20240426 13:12:28.680737   398 webserver.cc:460] Webserver started at http://127.0.99.190:35267/ using document root <none> and password file <none>
I20240426 13:12:28.680871   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:28.680917   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:28.681013   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:28.681360   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "12d1c316e88f445c8ff4de242ba46d28"
format_stamp: "Formatted at 2024-04-26 13:12:28 on dist-test-slave-2jsl"
I20240426 13:12:28.682703   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:12:28.683503  2029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.683740   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:28.683800   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "12d1c316e88f445c8ff4de242ba46d28"
format_stamp: "Formatted at 2024-04-26 13:12:28 on dist-test-slave-2jsl"
I20240426 13:12:28.683864   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:28.694631   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:28.694882   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:28.698446   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:43483
I20240426 13:12:28.704159  2081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:43483 every 8 connection(s)
I20240426 13:12:28.704545  2082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:28.706248  2082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28: Bootstrap starting.
I20240426 13:12:28.706955  2082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:28.707798  2082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28: No bootstrap required, opened a new log
I20240426 13:12:28.708086  2082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d1c316e88f445c8ff4de242ba46d28" member_type: VOTER }
I20240426 13:12:28.708161  2082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:28.708190  2082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d1c316e88f445c8ff4de242ba46d28, State: Initialized, Role: FOLLOWER
I20240426 13:12:28.708303  2082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d1c316e88f445c8ff4de242ba46d28" member_type: VOTER }
I20240426 13:12:28.708356  2082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:12:28.708387  2082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:12:28.708427  2082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:28.708982  2082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d1c316e88f445c8ff4de242ba46d28" member_type: VOTER }
I20240426 13:12:28.709090  2082 leader_election.cc:302] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d1c316e88f445c8ff4de242ba46d28; no voters: 
I20240426 13:12:28.709242  2082 leader_election.cc:288] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:12:28.709336  2085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:28.709522  2085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 1 LEADER]: Becoming Leader. State: Replica: 12d1c316e88f445c8ff4de242ba46d28, State: Running, Role: LEADER
I20240426 13:12:28.709647  2085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d1c316e88f445c8ff4de242ba46d28" member_type: VOTER }
I20240426 13:12:28.709721  2082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:12:28.710041  2087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d1c316e88f445c8ff4de242ba46d28. Latest consensus state: current_term: 1 leader_uuid: "12d1c316e88f445c8ff4de242ba46d28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d1c316e88f445c8ff4de242ba46d28" member_type: VOTER } }
I20240426 13:12:28.710026  2086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d1c316e88f445c8ff4de242ba46d28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d1c316e88f445c8ff4de242ba46d28" member_type: VOTER } }
I20240426 13:12:28.710155  2087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:28.710239  2086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:28.710647  2090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:12:28.711306  2090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:12:28.711453   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:12:28.712792  2090 catalog_manager.cc:1347] Generated new cluster ID: dff09caa8ced4f4b93e5c03dca93083a
I20240426 13:12:28.712844  2090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:12:28.730778  2090 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:12:28.731220  2090 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:12:28.742189  2090 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28: Generated new TSK 0
I20240426 13:12:28.742324  2090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:12:28.743517   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:28.746995  2104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:28.747850  2106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:28.748620   398 server_base.cc:1034] running on GCE node
W20240426 13:12:28.748651  2103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:28.748858   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:28.748895   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:28.748909   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137148748909 us; error 0 us; skew 500 ppm
I20240426 13:12:28.749743   398 webserver.cc:460] Webserver started at http://127.0.99.129:41589/ using document root <none> and password file <none>
I20240426 13:12:28.749914   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:28.749959   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:28.750057   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:28.750464   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "87e61d6cc7b549c8aeba305fd265d22f"
format_stamp: "Formatted at 2024-04-26 13:12:28 on dist-test-slave-2jsl"
I20240426 13:12:28.751837   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:12:28.752599  2111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.752794   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:12:28.752852   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "87e61d6cc7b549c8aeba305fd265d22f"
format_stamp: "Formatted at 2024-04-26 13:12:28 on dist-test-slave-2jsl"
I20240426 13:12:28.752912   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:28.765600   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:28.765898   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:28.766187   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:28.766589   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:28.766628   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.766672   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:28.766716   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.770251   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:44797
I20240426 13:12:28.770275  2174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:44797 every 8 connection(s)
I20240426 13:12:28.770922   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:28.773453  2180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:28.774175  2179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:28.774781  2175 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43483
I20240426 13:12:28.774884  2175 heartbeater.cc:464] Registering TS with master...
W20240426 13:12:28.774937  2182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:28.775094  2175 heartbeater.cc:511] Master 127.0.99.190:43483 requested a full tablet report, sending...
I20240426 13:12:28.775655  2046 ts_manager.cc:194] Registered new tserver with Master: 87e61d6cc7b549c8aeba305fd265d22f (127.0.99.129:44797)
I20240426 13:12:28.775713   398 server_base.cc:1034] running on GCE node
I20240426 13:12:28.775890   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:28.775926   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:28.775941   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137148775941 us; error 0 us; skew 500 ppm
I20240426 13:12:28.776386  2046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33882
I20240426 13:12:28.776794   398 webserver.cc:460] Webserver started at http://127.0.99.130:36785/ using document root <none> and password file <none>
I20240426 13:12:28.776945   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:28.776994   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:28.777050   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:28.777416   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "f2ee03ac83bb496980ff58ff83f622bc"
format_stamp: "Formatted at 2024-04-26 13:12:28 on dist-test-slave-2jsl"
I20240426 13:12:28.778728   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.001s
I20240426 13:12:28.779539  2187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.779731   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:28.779799   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "f2ee03ac83bb496980ff58ff83f622bc"
format_stamp: "Formatted at 2024-04-26 13:12:28 on dist-test-slave-2jsl"
I20240426 13:12:28.779857   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:28.793011   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:28.793265   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:28.793489   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:28.793843   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:28.793879   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.793921   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:28.793949   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.797515   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:43691
I20240426 13:12:28.797571  2250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:43691 every 8 connection(s)
I20240426 13:12:28.798224   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:28.800686  2254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:28.801369  2251 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43483
I20240426 13:12:28.801473  2251 heartbeater.cc:464] Registering TS with master...
W20240426 13:12:28.801476  2255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:28.801657  2251 heartbeater.cc:511] Master 127.0.99.190:43483 requested a full tablet report, sending...
I20240426 13:12:28.802232  2046 ts_manager.cc:194] Registered new tserver with Master: f2ee03ac83bb496980ff58ff83f622bc (127.0.99.130:43691)
I20240426 13:12:28.802266   398 server_base.cc:1034] running on GCE node
I20240426 13:12:28.802747  2046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33888
W20240426 13:12:28.803007  2257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:28.803202   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:28.803244   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:28.803265   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137148803265 us; error 0 us; skew 500 ppm
I20240426 13:12:28.803988   398 webserver.cc:460] Webserver started at http://127.0.99.131:43603/ using document root <none> and password file <none>
I20240426 13:12:28.804111   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:28.804152   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:28.804217   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:28.804538   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "0ec1afe2b285482baa33fdc2e34dd001"
format_stamp: "Formatted at 2024-04-26 13:12:28 on dist-test-slave-2jsl"
I20240426 13:12:28.805837   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:12:28.806671  2262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.806902   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:28.806962   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "0ec1afe2b285482baa33fdc2e34dd001"
format_stamp: "Formatted at 2024-04-26 13:12:28 on dist-test-slave-2jsl"
I20240426 13:12:28.807021   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestBeginCommitAfterFinalize.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:28.819111   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:28.819366   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:28.819591   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:28.819921   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:28.819953   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.820004   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:28.820050   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:28.823884   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:36723
I20240426 13:12:28.823944  2325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:36723 every 8 connection(s)
I20240426 13:12:28.827464  2326 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43483
I20240426 13:12:28.827543  2326 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:28.827697  2326 heartbeater.cc:511] Master 127.0.99.190:43483 requested a full tablet report, sending...
I20240426 13:12:28.828258  2046 ts_manager.cc:194] Registered new tserver with Master: 0ec1afe2b285482baa33fdc2e34dd001 (127.0.99.131:36723)
I20240426 13:12:28.828681   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004456501s
I20240426 13:12:28.828708  2046 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33904
I20240426 13:12:28.829171   398 test_util.cc:268] Using random seed: -1478873615
I20240426 13:12:28.835037  2046 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33912:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:12:28.835373  2046 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:12:28.843441  2139 tablet_service.cc:1467] Processing CreateTablet for tablet 25e824abd9354312aa23c61ba236f091 (DEFAULT_TABLE table=test-workload [id=fd94070c7efb45628303cb9063134e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:28.843595  2215 tablet_service.cc:1467] Processing CreateTablet for tablet 25e824abd9354312aa23c61ba236f091 (DEFAULT_TABLE table=test-workload [id=fd94070c7efb45628303cb9063134e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:28.843683  2139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e824abd9354312aa23c61ba236f091. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:28.843684  2290 tablet_service.cc:1467] Processing CreateTablet for tablet 25e824abd9354312aa23c61ba236f091 (DEFAULT_TABLE table=test-workload [id=fd94070c7efb45628303cb9063134e30]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:28.843856  2215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e824abd9354312aa23c61ba236f091. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:28.843950  2290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e824abd9354312aa23c61ba236f091. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:28.845577  2341 tablet_bootstrap.cc:492] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc: Bootstrap starting.
I20240426 13:12:28.845820  2342 tablet_bootstrap.cc:492] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f: Bootstrap starting.
I20240426 13:12:28.846626  2341 tablet_bootstrap.cc:654] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:28.846626  2342 tablet_bootstrap.cc:654] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:28.846777  2343 tablet_bootstrap.cc:492] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001: Bootstrap starting.
I20240426 13:12:28.847535  2343 tablet_bootstrap.cc:654] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:28.847564  2342 tablet_bootstrap.cc:492] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f: No bootstrap required, opened a new log
I20240426 13:12:28.847658  2341 tablet_bootstrap.cc:492] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc: No bootstrap required, opened a new log
I20240426 13:12:28.847656  2342 ts_tablet_manager.cc:1367] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:12:28.847755  2341 ts_tablet_manager.cc:1367] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:12:28.848037  2342 raft_consensus.cc:348] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:28.848126  2342 raft_consensus.cc:374] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:28.848130  2341 raft_consensus.cc:348] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:28.848309  2341 raft_consensus.cc:374] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:28.848348  2341 raft_consensus.cc:729] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ee03ac83bb496980ff58ff83f622bc, State: Initialized, Role: FOLLOWER
I20240426 13:12:28.848460  2341 consensus_queue.cc:260] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:28.848570  2343 tablet_bootstrap.cc:492] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001: No bootstrap required, opened a new log
I20240426 13:12:28.848636  2343 ts_tablet_manager.cc:1367] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:28.848753  2251 heartbeater.cc:503] Master 127.0.99.190:43483 was elected leader, sending a full tablet report...
I20240426 13:12:28.848170  2342 raft_consensus.cc:729] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e61d6cc7b549c8aeba305fd265d22f, State: Initialized, Role: FOLLOWER
I20240426 13:12:28.848910  2342 consensus_queue.cc:260] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:28.849083  2343 raft_consensus.cc:348] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:28.849161  2343 raft_consensus.cc:374] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:28.849190  2343 raft_consensus.cc:729] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec1afe2b285482baa33fdc2e34dd001, State: Initialized, Role: FOLLOWER
I20240426 13:12:28.849195  2342 ts_tablet_manager.cc:1398] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:28.849289  2343 consensus_queue.cc:260] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:28.849371  2175 heartbeater.cc:503] Master 127.0.99.190:43483 was elected leader, sending a full tablet report...
I20240426 13:12:28.849603  2343 ts_tablet_manager.cc:1398] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:28.849720  2326 heartbeater.cc:503] Master 127.0.99.190:43483 was elected leader, sending a full tablet report...
I20240426 13:12:28.848775  2341 ts_tablet_manager.cc:1398] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:29.007483  2348 raft_consensus.cc:482] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:12:29.007635  2348 raft_consensus.cc:504] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:29.008131  2348 leader_election.cc:288] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2ee03ac83bb496980ff58ff83f622bc (127.0.99.130:43691), 0ec1afe2b285482baa33fdc2e34dd001 (127.0.99.131:36723)
I20240426 13:12:29.011883  2225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e824abd9354312aa23c61ba236f091" candidate_uuid: "87e61d6cc7b549c8aeba305fd265d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2ee03ac83bb496980ff58ff83f622bc" is_pre_election: true
I20240426 13:12:29.011950  2300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e824abd9354312aa23c61ba236f091" candidate_uuid: "87e61d6cc7b549c8aeba305fd265d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec1afe2b285482baa33fdc2e34dd001" is_pre_election: true
I20240426 13:12:29.011998  2225 raft_consensus.cc:2454] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e61d6cc7b549c8aeba305fd265d22f in term 0.
I20240426 13:12:29.012064  2300 raft_consensus.cc:2454] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e61d6cc7b549c8aeba305fd265d22f in term 0.
I20240426 13:12:29.012391  2114 leader_election.cc:302] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec1afe2b285482baa33fdc2e34dd001, 87e61d6cc7b549c8aeba305fd265d22f; no voters: 
I20240426 13:12:29.012549  2348 raft_consensus.cc:2781] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:12:29.012607  2348 raft_consensus.cc:482] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:12:29.012634  2348 raft_consensus.cc:3037] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:29.013422  2348 raft_consensus.cc:504] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:29.013700  2348 leader_election.cc:288] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [CANDIDATE]: Term 1 election: Requested vote from peers f2ee03ac83bb496980ff58ff83f622bc (127.0.99.130:43691), 0ec1afe2b285482baa33fdc2e34dd001 (127.0.99.131:36723)
I20240426 13:12:29.013984  2225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e824abd9354312aa23c61ba236f091" candidate_uuid: "87e61d6cc7b549c8aeba305fd265d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2ee03ac83bb496980ff58ff83f622bc"
I20240426 13:12:29.014007  2300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e824abd9354312aa23c61ba236f091" candidate_uuid: "87e61d6cc7b549c8aeba305fd265d22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec1afe2b285482baa33fdc2e34dd001"
I20240426 13:12:29.014091  2225 raft_consensus.cc:3037] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:29.014094  2300 raft_consensus.cc:3037] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:29.014974  2225 raft_consensus.cc:2454] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e61d6cc7b549c8aeba305fd265d22f in term 1.
I20240426 13:12:29.014974  2300 raft_consensus.cc:2454] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e61d6cc7b549c8aeba305fd265d22f in term 1.
I20240426 13:12:29.015234  2114 leader_election.cc:302] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec1afe2b285482baa33fdc2e34dd001, 87e61d6cc7b549c8aeba305fd265d22f; no voters: 
I20240426 13:12:29.015408  2348 raft_consensus.cc:2781] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:29.015641  2348 raft_consensus.cc:686] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 1 LEADER]: Becoming Leader. State: Replica: 87e61d6cc7b549c8aeba305fd265d22f, State: Running, Role: LEADER
I20240426 13:12:29.015792  2348 consensus_queue.cc:237] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } }
I20240426 13:12:29.017204  2046 catalog_manager.cc:5484] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f reported cstate change: term changed from 0 to 1, leader changed from <none> to 87e61d6cc7b549c8aeba305fd265d22f (127.0.99.129). New cstate: current_term: 1 leader_uuid: "87e61d6cc7b549c8aeba305fd265d22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e61d6cc7b549c8aeba305fd265d22f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 44797 } health_report { overall_health: HEALTHY } } }
W20240426 13:12:29.020809  2176 tablet.cc:2369] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 13:12:29.048619  2225 raft_consensus.cc:1261] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 1 FOLLOWER]: Refusing update from remote peer 87e61d6cc7b549c8aeba305fd265d22f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:29.048619  2300 raft_consensus.cc:1261] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 1 FOLLOWER]: Refusing update from remote peer 87e61d6cc7b549c8aeba305fd265d22f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:29.049031  2348 consensus_queue.cc:1035] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec1afe2b285482baa33fdc2e34dd001" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 36723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:29.049185  2351 consensus_queue.cc:1035] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ee03ac83bb496980ff58ff83f622bc" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240426 13:12:39.053295  2358 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:40092: Illegal state: Transaction 0 commit already in progress
I20240426 13:12:39.053719   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:12:39.056429   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:39.056557   398 tablet_replica.cc:330] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f: stopping tablet replica
I20240426 13:12:39.056649   398 raft_consensus.cc:2229] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:39.056766   398 raft_consensus.cc:2258] T 25e824abd9354312aa23c61ba236f091 P 87e61d6cc7b549c8aeba305fd265d22f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:39.072429   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:12:39.074128   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:12:39.077737   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:39.077891   398 tablet_replica.cc:330] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc: stopping tablet replica
I20240426 13:12:39.077998   398 raft_consensus.cc:2229] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:39.078084   398 raft_consensus.cc:2258] T 25e824abd9354312aa23c61ba236f091 P f2ee03ac83bb496980ff58ff83f622bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:39.094596   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:12:39.095968   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:12:39.098830   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:39.098956   398 tablet_replica.cc:330] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001: stopping tablet replica
I20240426 13:12:39.099042   398 raft_consensus.cc:2229] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:39.099121   398 raft_consensus.cc:2258] T 25e824abd9354312aa23c61ba236f091 P 0ec1afe2b285482baa33fdc2e34dd001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:39.103976   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:12:39.105281   398 master.cc:537] Master@127.0.99.190:43483 shutting down...
I20240426 13:12:39.107959   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:39.108054   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:39.108106   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 12d1c316e88f445c8ff4de242ba46d28: stopping tablet replica
I20240426 13:12:39.119891   398 master.cc:552] Master@127.0.99.190:43483 shutdown complete.
[       OK ] TxnParticipantITest.TestBeginCommitAfterFinalize (10454 ms)
[ RUN      ] TxnParticipantITest.TestProxyErrorWhenNotBegun
I20240426 13:12:39.128180   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:35909
I20240426 13:12:39.128453   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:39.131040  2404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:39.131892  2405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:39.132910   398 server_base.cc:1034] running on GCE node
W20240426 13:12:39.133595  2407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:39.133811   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:39.133857   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:39.133880   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137159133879 us; error 0 us; skew 500 ppm
I20240426 13:12:39.134744   398 webserver.cc:460] Webserver started at http://127.0.99.190:43937/ using document root <none> and password file <none>
I20240426 13:12:39.134873   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:39.134915   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:39.134984   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:39.135308   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "94e81bc0214c4ce88374ea5ab5f604a3"
format_stamp: "Formatted at 2024-04-26 13:12:39 on dist-test-slave-2jsl"
I20240426 13:12:39.136574   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:12:39.137655  2412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.137952   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:39.138164   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "94e81bc0214c4ce88374ea5ab5f604a3"
format_stamp: "Formatted at 2024-04-26 13:12:39 on dist-test-slave-2jsl"
I20240426 13:12:39.138226   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:39.151645   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:39.151923   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:39.155678   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:35909
I20240426 13:12:39.159998  2464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:35909 every 8 connection(s)
I20240426 13:12:39.160409  2465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:39.161865  2465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3: Bootstrap starting.
I20240426 13:12:39.162549  2465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:39.163357  2465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3: No bootstrap required, opened a new log
I20240426 13:12:39.163661  2465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e81bc0214c4ce88374ea5ab5f604a3" member_type: VOTER }
I20240426 13:12:39.163756  2465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:39.163779  2465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94e81bc0214c4ce88374ea5ab5f604a3, State: Initialized, Role: FOLLOWER
I20240426 13:12:39.163882  2465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e81bc0214c4ce88374ea5ab5f604a3" member_type: VOTER }
I20240426 13:12:39.163957  2465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:12:39.163988  2465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:12:39.164026  2465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:39.164593  2465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e81bc0214c4ce88374ea5ab5f604a3" member_type: VOTER }
I20240426 13:12:39.164695  2465 leader_election.cc:302] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94e81bc0214c4ce88374ea5ab5f604a3; no voters: 
I20240426 13:12:39.164840  2465 leader_election.cc:288] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:12:39.164943  2468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:39.165104  2468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 1 LEADER]: Becoming Leader. State: Replica: 94e81bc0214c4ce88374ea5ab5f604a3, State: Running, Role: LEADER
I20240426 13:12:39.165254  2465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:12:39.165249  2468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e81bc0214c4ce88374ea5ab5f604a3" member_type: VOTER }
I20240426 13:12:39.165638  2469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94e81bc0214c4ce88374ea5ab5f604a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e81bc0214c4ce88374ea5ab5f604a3" member_type: VOTER } }
I20240426 13:12:39.165665  2470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94e81bc0214c4ce88374ea5ab5f604a3. Latest consensus state: current_term: 1 leader_uuid: "94e81bc0214c4ce88374ea5ab5f604a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94e81bc0214c4ce88374ea5ab5f604a3" member_type: VOTER } }
I20240426 13:12:39.165784  2469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:39.165865  2470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:39.166301  2474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:12:39.166985  2474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:12:39.167174   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:12:39.168555  2474 catalog_manager.cc:1347] Generated new cluster ID: 32cb5fd70cac40bc88fd8122a2c44f32
I20240426 13:12:39.168605  2474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:12:39.181017  2474 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:12:39.181524  2474 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:12:39.186983  2474 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3: Generated new TSK 0
I20240426 13:12:39.187126  2474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:12:39.199164   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:39.201622  2487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:39.202364  2486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:39.203225  2489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:39.203936   398 server_base.cc:1034] running on GCE node
I20240426 13:12:39.204048   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:39.204078   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:39.204090   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137159204090 us; error 0 us; skew 500 ppm
I20240426 13:12:39.204851   398 webserver.cc:460] Webserver started at http://127.0.99.129:42421/ using document root <none> and password file <none>
I20240426 13:12:39.204969   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:39.205013   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:39.205077   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:39.205442   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "0bf8538b1d804c57ab69538c01692a2f"
format_stamp: "Formatted at 2024-04-26 13:12:39 on dist-test-slave-2jsl"
I20240426 13:12:39.206902   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:12:39.207692  2494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.207926   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:39.207978   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "0bf8538b1d804c57ab69538c01692a2f"
format_stamp: "Formatted at 2024-04-26 13:12:39 on dist-test-slave-2jsl"
I20240426 13:12:39.208037   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:39.216148   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:39.216380   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:39.216572   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:39.216958   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:39.216993   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.217021   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:39.217056   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.220424   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:40669
I20240426 13:12:39.220444  2557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:40669 every 8 connection(s)
I20240426 13:12:39.221040   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:39.223364  2563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:39.223978  2562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:39.224684  2565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:39.224786  2558 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35909
I20240426 13:12:39.224865  2558 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:39.225015  2558 heartbeater.cc:511] Master 127.0.99.190:35909 requested a full tablet report, sending...
I20240426 13:12:39.225378   398 server_base.cc:1034] running on GCE node
I20240426 13:12:39.225507   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:39.225539   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:39.225549  2429 ts_manager.cc:194] Registered new tserver with Master: 0bf8538b1d804c57ab69538c01692a2f (127.0.99.129:40669)
I20240426 13:12:39.225557   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137159225557 us; error 0 us; skew 500 ppm
I20240426 13:12:39.226222  2429 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58028
I20240426 13:12:39.226452   398 webserver.cc:460] Webserver started at http://127.0.99.130:33591/ using document root <none> and password file <none>
I20240426 13:12:39.226610   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:39.226670   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:39.226722   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:39.227044   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "235e32d8aec7462caaa2fc6a381bb793"
format_stamp: "Formatted at 2024-04-26 13:12:39 on dist-test-slave-2jsl"
I20240426 13:12:39.228283   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:39.228965  2570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.229149   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:12:39.229204   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "235e32d8aec7462caaa2fc6a381bb793"
format_stamp: "Formatted at 2024-04-26 13:12:39 on dist-test-slave-2jsl"
I20240426 13:12:39.229261   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:39.244467   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:39.244709   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:39.244925   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:39.245255   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:39.245290   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.245325   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:39.245357   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.249017   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:35681
I20240426 13:12:39.249061  2633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:35681 every 8 connection(s)
I20240426 13:12:39.249648   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:39.251945  2637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:39.252712  2638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:39.252988  2634 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35909
I20240426 13:12:39.253062  2634 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:39.253225  2634 heartbeater.cc:511] Master 127.0.99.190:35909 requested a full tablet report, sending...
W20240426 13:12:39.253444  2640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:39.253772  2429 ts_manager.cc:194] Registered new tserver with Master: 235e32d8aec7462caaa2fc6a381bb793 (127.0.99.130:35681)
I20240426 13:12:39.254192   398 server_base.cc:1034] running on GCE node
I20240426 13:12:39.254334  2429 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58044
I20240426 13:12:39.254334   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:39.254413   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:39.254441   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137159254440 us; error 0 us; skew 500 ppm
I20240426 13:12:39.255198   398 webserver.cc:460] Webserver started at http://127.0.99.131:35325/ using document root <none> and password file <none>
I20240426 13:12:39.255318   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:39.255359   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:39.255434   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:39.255798   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "03df5eec670a4f048e3e087d070d3027"
format_stamp: "Formatted at 2024-04-26 13:12:39 on dist-test-slave-2jsl"
I20240426 13:12:39.257054   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:39.257845  2645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.258073   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:39.258131   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "03df5eec670a4f048e3e087d070d3027"
format_stamp: "Formatted at 2024-04-26 13:12:39 on dist-test-slave-2jsl"
I20240426 13:12:39.258189   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:39.285496   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:39.285797   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:39.286042   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:39.286437   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:39.286473   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.286516   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:39.286545   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:39.290925   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:32775
I20240426 13:12:39.290971  2708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:32775 every 8 connection(s)
I20240426 13:12:39.294399  2709 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35909
I20240426 13:12:39.294467  2709 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:39.294615  2709 heartbeater.cc:511] Master 127.0.99.190:35909 requested a full tablet report, sending...
I20240426 13:12:39.295092  2429 ts_manager.cc:194] Registered new tserver with Master: 03df5eec670a4f048e3e087d070d3027 (127.0.99.131:32775)
I20240426 13:12:39.295516  2429 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58048
I20240426 13:12:39.295758   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.0044747s
I20240426 13:12:39.296237   398 test_util.cc:268] Using random seed: -1468406548
I20240426 13:12:39.302229  2429 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58050:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:12:39.302548  2429 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:12:39.310143  2522 tablet_service.cc:1467] Processing CreateTablet for tablet b2a7f6955faf46b89906346d4c240b23 (DEFAULT_TABLE table=test-workload [id=49a8a83808974b59bfda097b44a375a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:39.310163  2673 tablet_service.cc:1467] Processing CreateTablet for tablet b2a7f6955faf46b89906346d4c240b23 (DEFAULT_TABLE table=test-workload [id=49a8a83808974b59bfda097b44a375a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:39.310380  2522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2a7f6955faf46b89906346d4c240b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:39.310385  2673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2a7f6955faf46b89906346d4c240b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:39.310777  2598 tablet_service.cc:1467] Processing CreateTablet for tablet b2a7f6955faf46b89906346d4c240b23 (DEFAULT_TABLE table=test-workload [id=49a8a83808974b59bfda097b44a375a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:39.311025  2598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b2a7f6955faf46b89906346d4c240b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:39.312294  2724 tablet_bootstrap.cc:492] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027: Bootstrap starting.
I20240426 13:12:39.313019  2725 tablet_bootstrap.cc:492] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f: Bootstrap starting.
I20240426 13:12:39.313124  2724 tablet_bootstrap.cc:654] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:39.313681  2725 tablet_bootstrap.cc:654] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:39.313939  2724 tablet_bootstrap.cc:492] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027: No bootstrap required, opened a new log
I20240426 13:12:39.314033  2724 ts_tablet_manager.cc:1367] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:39.314220  2726 tablet_bootstrap.cc:492] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793: Bootstrap starting.
I20240426 13:12:39.314340  2724 raft_consensus.cc:348] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.314450  2724 raft_consensus.cc:374] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:39.314512  2724 raft_consensus.cc:729] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03df5eec670a4f048e3e087d070d3027, State: Initialized, Role: FOLLOWER
I20240426 13:12:39.314664  2725 tablet_bootstrap.cc:492] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f: No bootstrap required, opened a new log
I20240426 13:12:39.314635  2724 consensus_queue.cc:260] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.314745  2725 ts_tablet_manager.cc:1367] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:39.314977  2709 heartbeater.cc:503] Master 127.0.99.190:35909 was elected leader, sending a full tablet report...
I20240426 13:12:39.315004  2726 tablet_bootstrap.cc:654] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:39.315097  2725 raft_consensus.cc:348] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.315178  2725 raft_consensus.cc:374] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:39.315245  2725 raft_consensus.cc:729] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bf8538b1d804c57ab69538c01692a2f, State: Initialized, Role: FOLLOWER
I20240426 13:12:39.315397  2724 ts_tablet_manager.cc:1398] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:39.315380  2725 consensus_queue.cc:260] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.315749  2725 ts_tablet_manager.cc:1398] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:39.315754  2558 heartbeater.cc:503] Master 127.0.99.190:35909 was elected leader, sending a full tablet report...
I20240426 13:12:39.316828  2726 tablet_bootstrap.cc:492] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793: No bootstrap required, opened a new log
I20240426 13:12:39.316917  2726 ts_tablet_manager.cc:1367] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:12:39.317271  2726 raft_consensus.cc:348] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.317376  2726 raft_consensus.cc:374] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:39.317416  2726 raft_consensus.cc:729] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235e32d8aec7462caaa2fc6a381bb793, State: Initialized, Role: FOLLOWER
I20240426 13:12:39.317524  2726 consensus_queue.cc:260] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.317873  2634 heartbeater.cc:503] Master 127.0.99.190:35909 was elected leader, sending a full tablet report...
I20240426 13:12:39.317878  2726 ts_tablet_manager.cc:1398] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:39.319600  2731 raft_consensus.cc:482] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:12:39.319706  2731 raft_consensus.cc:504] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.320068  2731 leader_election.cc:288] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03df5eec670a4f048e3e087d070d3027 (127.0.99.131:32775), 235e32d8aec7462caaa2fc6a381bb793 (127.0.99.130:35681)
I20240426 13:12:39.323235  2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2a7f6955faf46b89906346d4c240b23" candidate_uuid: "0bf8538b1d804c57ab69538c01692a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03df5eec670a4f048e3e087d070d3027" is_pre_election: true
I20240426 13:12:39.323330  2683 raft_consensus.cc:2454] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf8538b1d804c57ab69538c01692a2f in term 0.
I20240426 13:12:39.323539  2608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2a7f6955faf46b89906346d4c240b23" candidate_uuid: "0bf8538b1d804c57ab69538c01692a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235e32d8aec7462caaa2fc6a381bb793" is_pre_election: true
I20240426 13:12:39.323582  2498 leader_election.cc:302] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03df5eec670a4f048e3e087d070d3027, 0bf8538b1d804c57ab69538c01692a2f; no voters: 
I20240426 13:12:39.323637  2608 raft_consensus.cc:2454] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bf8538b1d804c57ab69538c01692a2f in term 0.
I20240426 13:12:39.323772  2731 raft_consensus.cc:2781] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:12:39.323822  2731 raft_consensus.cc:482] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:12:39.323868  2731 raft_consensus.cc:3037] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:39.324514  2731 raft_consensus.cc:504] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.324769  2731 leader_election.cc:288] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [CANDIDATE]: Term 1 election: Requested vote from peers 03df5eec670a4f048e3e087d070d3027 (127.0.99.131:32775), 235e32d8aec7462caaa2fc6a381bb793 (127.0.99.130:35681)
I20240426 13:12:39.324973  2683 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2a7f6955faf46b89906346d4c240b23" candidate_uuid: "0bf8538b1d804c57ab69538c01692a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03df5eec670a4f048e3e087d070d3027"
I20240426 13:12:39.325021  2608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b2a7f6955faf46b89906346d4c240b23" candidate_uuid: "0bf8538b1d804c57ab69538c01692a2f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235e32d8aec7462caaa2fc6a381bb793"
I20240426 13:12:39.325055  2683 raft_consensus.cc:3037] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:39.325127  2608 raft_consensus.cc:3037] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:39.325788  2683 raft_consensus.cc:2454] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf8538b1d804c57ab69538c01692a2f in term 1.
I20240426 13:12:39.325857  2608 raft_consensus.cc:2454] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bf8538b1d804c57ab69538c01692a2f in term 1.
I20240426 13:12:39.326036  2498 leader_election.cc:302] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03df5eec670a4f048e3e087d070d3027, 0bf8538b1d804c57ab69538c01692a2f; no voters: 
I20240426 13:12:39.326195  2731 raft_consensus.cc:2781] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:39.326375  2731 raft_consensus.cc:686] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 1 LEADER]: Becoming Leader. State: Replica: 0bf8538b1d804c57ab69538c01692a2f, State: Running, Role: LEADER
I20240426 13:12:39.326534  2731 consensus_queue.cc:237] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } }
I20240426 13:12:39.327728  2429 catalog_manager.cc:5484] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bf8538b1d804c57ab69538c01692a2f (127.0.99.129). New cstate: current_term: 1 leader_uuid: "0bf8538b1d804c57ab69538c01692a2f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0bf8538b1d804c57ab69538c01692a2f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 } health_report { overall_health: UNKNOWN } } }
I20240426 13:12:39.355865  2608 raft_consensus.cc:1261] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 1 FOLLOWER]: Refusing update from remote peer 0bf8538b1d804c57ab69538c01692a2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:39.355890  2683 raft_consensus.cc:1261] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 1 FOLLOWER]: Refusing update from remote peer 0bf8538b1d804c57ab69538c01692a2f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:39.356209  2731 consensus_queue.cc:1035] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "235e32d8aec7462caaa2fc6a381bb793" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:39.356323  2734 consensus_queue.cc:1035] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [LEADER]: Connected to new peer: Peer: permanent_uuid: "03df5eec670a4f048e3e087d070d3027" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:39.360468  2742 mvcc.cc:205] Tried to move back new op lower bound from 7021105804721868800 to 7021105804602769408. Current Snapshot: MvccSnapshot[applied={T|T < 7021105804718796800}]
W20240426 13:12:49.354622  2741 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53932: Illegal state: Transaction hasn't been successfully started
W20240426 13:12:49.355454  2741 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:53932: Illegal state: Transaction metadata for transaction 1 does not exist
I20240426 13:12:49.358129   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:12:49.361585   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:49.361733   398 tablet_replica.cc:330] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f: stopping tablet replica
I20240426 13:12:49.361831   398 raft_consensus.cc:2229] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:49.361995   398 raft_consensus.cc:2258] T b2a7f6955faf46b89906346d4c240b23 P 0bf8538b1d804c57ab69538c01692a2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:49.377981   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:12:49.379619   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:12:49.382643   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:49.382767   398 tablet_replica.cc:330] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793: stopping tablet replica
I20240426 13:12:49.382856   398 raft_consensus.cc:2229] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:49.382933   398 raft_consensus.cc:2258] T b2a7f6955faf46b89906346d4c240b23 P 235e32d8aec7462caaa2fc6a381bb793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:49.387614   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:12:49.389025   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:12:49.391851   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:49.391988   398 tablet_replica.cc:330] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027: stopping tablet replica
I20240426 13:12:49.392077   398 raft_consensus.cc:2229] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:49.392170   398 raft_consensus.cc:2258] T b2a7f6955faf46b89906346d4c240b23 P 03df5eec670a4f048e3e087d070d3027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:49.411114   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:12:49.412705   398 master.cc:537] Master@127.0.99.190:35909 shutting down...
I20240426 13:12:49.415328   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:49.415433   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:49.415495   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 94e81bc0214c4ce88374ea5ab5f604a3: stopping tablet replica
I20240426 13:12:49.427354   398 master.cc:552] Master@127.0.99.190:35909 shutdown complete.
[       OK ] TxnParticipantITest.TestProxyErrorWhenNotBegun (10306 ms)
[ RUN      ] TxnParticipantITest.TestProxyIllegalStatesInCommitSequence
I20240426 13:12:49.435046   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:40043
I20240426 13:12:49.435300   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:49.437819  2783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:49.438819   398 server_base.cc:1034] running on GCE node
W20240426 13:12:49.439643  2784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:49.440459  2786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:49.440661   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:49.440699   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:49.440714   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137169440714 us; error 0 us; skew 500 ppm
I20240426 13:12:49.441519   398 webserver.cc:460] Webserver started at http://127.0.99.190:46631/ using document root <none> and password file <none>
I20240426 13:12:49.441627   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:49.441659   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:49.441699   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:49.442066   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "e10ce95ba70247a791dcd5130802b253"
format_stamp: "Formatted at 2024-04-26 13:12:49 on dist-test-slave-2jsl"
I20240426 13:12:49.443243   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:49.443986  2791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.444195   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:49.444269   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "e10ce95ba70247a791dcd5130802b253"
format_stamp: "Formatted at 2024-04-26 13:12:49 on dist-test-slave-2jsl"
I20240426 13:12:49.444330   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:49.480341   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:49.480592   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:49.484043   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:40043
I20240426 13:12:49.486994  2843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:40043 every 8 connection(s)
I20240426 13:12:49.487422  2844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:49.488903  2844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253: Bootstrap starting.
I20240426 13:12:49.489509  2844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:49.490288  2844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253: No bootstrap required, opened a new log
I20240426 13:12:49.490563  2844 raft_consensus.cc:348] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e10ce95ba70247a791dcd5130802b253" member_type: VOTER }
I20240426 13:12:49.490634  2844 raft_consensus.cc:374] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:49.490654  2844 raft_consensus.cc:729] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e10ce95ba70247a791dcd5130802b253, State: Initialized, Role: FOLLOWER
I20240426 13:12:49.490763  2844 consensus_queue.cc:260] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e10ce95ba70247a791dcd5130802b253" member_type: VOTER }
I20240426 13:12:49.490830  2844 raft_consensus.cc:388] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:12:49.490855  2844 raft_consensus.cc:482] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:12:49.490885  2844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:49.491408  2844 raft_consensus.cc:504] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e10ce95ba70247a791dcd5130802b253" member_type: VOTER }
I20240426 13:12:49.491497  2844 leader_election.cc:302] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e10ce95ba70247a791dcd5130802b253; no voters: 
I20240426 13:12:49.491611  2844 leader_election.cc:288] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:12:49.491712  2847 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:49.491868  2847 raft_consensus.cc:686] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 1 LEADER]: Becoming Leader. State: Replica: e10ce95ba70247a791dcd5130802b253, State: Running, Role: LEADER
I20240426 13:12:49.491931  2844 sys_catalog.cc:564] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:12:49.491993  2847 consensus_queue.cc:237] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e10ce95ba70247a791dcd5130802b253" member_type: VOTER }
I20240426 13:12:49.492364  2848 sys_catalog.cc:455] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e10ce95ba70247a791dcd5130802b253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e10ce95ba70247a791dcd5130802b253" member_type: VOTER } }
I20240426 13:12:49.492401  2849 sys_catalog.cc:455] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e10ce95ba70247a791dcd5130802b253. Latest consensus state: current_term: 1 leader_uuid: "e10ce95ba70247a791dcd5130802b253" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e10ce95ba70247a791dcd5130802b253" member_type: VOTER } }
I20240426 13:12:49.492519  2848 sys_catalog.cc:458] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:49.492609  2849 sys_catalog.cc:458] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:49.492977  2855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:12:49.493716  2855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:12:49.493901   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:12:49.495142  2855 catalog_manager.cc:1347] Generated new cluster ID: 8a84d777fc294223adda7527f40c9b21
I20240426 13:12:49.495186  2855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:12:49.507541  2855 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:12:49.508044  2855 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:12:49.513337  2855 catalog_manager.cc:5857] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253: Generated new TSK 0
I20240426 13:12:49.513463  2855 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:12:49.526062   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:49.528331  2865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:49.529247  2866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:49.530026  2868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:49.530807   398 server_base.cc:1034] running on GCE node
I20240426 13:12:49.530910   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:49.530939   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:49.530952   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137169530952 us; error 0 us; skew 500 ppm
I20240426 13:12:49.531689   398 webserver.cc:460] Webserver started at http://127.0.99.129:45751/ using document root <none> and password file <none>
I20240426 13:12:49.531806   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:49.531839   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:49.531881   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:49.532156   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "5cb611a3376940f3b1b7134f38b8662a"
format_stamp: "Formatted at 2024-04-26 13:12:49 on dist-test-slave-2jsl"
I20240426 13:12:49.533412   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:49.534222  2873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.534427   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:49.534483   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "5cb611a3376940f3b1b7134f38b8662a"
format_stamp: "Formatted at 2024-04-26 13:12:49 on dist-test-slave-2jsl"
I20240426 13:12:49.534545   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:49.544732   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:49.544984   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:49.545212   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:49.545563   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:49.545598   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.545635   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:49.545666   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.549248   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:43393
I20240426 13:12:49.549271  2936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:43393 every 8 connection(s)
I20240426 13:12:49.549793   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:49.551996  2941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:49.552706  2944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:49.553459  2937 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40043
W20240426 13:12:49.553490  2942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:49.553547  2937 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:49.553732  2937 heartbeater.cc:511] Master 127.0.99.190:40043 requested a full tablet report, sending...
I20240426 13:12:49.554279   398 server_base.cc:1034] running on GCE node
I20240426 13:12:49.554309  2807 ts_manager.cc:194] Registered new tserver with Master: 5cb611a3376940f3b1b7134f38b8662a (127.0.99.129:43393)
I20240426 13:12:49.554457   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:49.554499   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:49.554519   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137169554519 us; error 0 us; skew 500 ppm
I20240426 13:12:49.555205  2807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42858
I20240426 13:12:49.555402   398 webserver.cc:460] Webserver started at http://127.0.99.130:33715/ using document root <none> and password file <none>
I20240426 13:12:49.555558   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:49.555590   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:49.555630   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:49.555902   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "5d80b796768a4ae5a5ae41cf7e45997a"
format_stamp: "Formatted at 2024-04-26 13:12:49 on dist-test-slave-2jsl"
I20240426 13:12:49.557087   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:49.557920  2949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.558177   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:49.558235   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "5d80b796768a4ae5a5ae41cf7e45997a"
format_stamp: "Formatted at 2024-04-26 13:12:49 on dist-test-slave-2jsl"
I20240426 13:12:49.558300   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:49.561820   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:49.562095   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:49.562312   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:49.562672   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:49.562706   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.562736   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:49.562767   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.566215   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:37275
I20240426 13:12:49.566236  3012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:37275 every 8 connection(s)
I20240426 13:12:49.566808   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:49.568971  3016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:49.569751  3017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:49.570421  3019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:49.570573  3013 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40043
I20240426 13:12:49.570634  3013 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:49.570775  3013 heartbeater.cc:511] Master 127.0.99.190:40043 requested a full tablet report, sending...
I20240426 13:12:49.571225   398 server_base.cc:1034] running on GCE node
I20240426 13:12:49.571238  2807 ts_manager.cc:194] Registered new tserver with Master: 5d80b796768a4ae5a5ae41cf7e45997a (127.0.99.130:37275)
I20240426 13:12:49.571404   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:49.571441   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:49.571458   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137169571459 us; error 0 us; skew 500 ppm
I20240426 13:12:49.571820  2807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42872
I20240426 13:12:49.572201   398 webserver.cc:460] Webserver started at http://127.0.99.131:43701/ using document root <none> and password file <none>
I20240426 13:12:49.572336   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:49.572381   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:49.572441   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:49.572719   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088"
format_stamp: "Formatted at 2024-04-26 13:12:49 on dist-test-slave-2jsl"
I20240426 13:12:49.573982   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:12:49.574991  3024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.575285   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:49.575362   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088"
format_stamp: "Formatted at 2024-04-26 13:12:49 on dist-test-slave-2jsl"
I20240426 13:12:49.575440   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:49.582232   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:49.582475   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:49.582716   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:49.583063   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:49.583098   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.583146   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:49.583174   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:49.587131   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:33491
I20240426 13:12:49.587157  3087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:33491 every 8 connection(s)
I20240426 13:12:49.590582  3088 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40043
I20240426 13:12:49.590644  3088 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:49.590801  3088 heartbeater.cc:511] Master 127.0.99.190:40043 requested a full tablet report, sending...
I20240426 13:12:49.591248  2807 ts_manager.cc:194] Registered new tserver with Master: 1c3ce3c63b3d42fea7ce7a9b11d17088 (127.0.99.131:33491)
I20240426 13:12:49.591650  2807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42874
I20240426 13:12:49.591907   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004457941s
I20240426 13:12:49.592296   398 test_util.cc:268] Using random seed: -1458110489
I20240426 13:12:49.598260  2807 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42884:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:12:49.598619  2807 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:12:49.606969  3052 tablet_service.cc:1467] Processing CreateTablet for tablet 4e6470e875614eec850a2fbafb26d548 (DEFAULT_TABLE table=test-workload [id=e9b2b3a0ad264472b96624b6d01b2cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:49.607231  3052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e6470e875614eec850a2fbafb26d548. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:49.607371  2901 tablet_service.cc:1467] Processing CreateTablet for tablet 4e6470e875614eec850a2fbafb26d548 (DEFAULT_TABLE table=test-workload [id=e9b2b3a0ad264472b96624b6d01b2cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:49.607601  2901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e6470e875614eec850a2fbafb26d548. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:49.607599  2977 tablet_service.cc:1467] Processing CreateTablet for tablet 4e6470e875614eec850a2fbafb26d548 (DEFAULT_TABLE table=test-workload [id=e9b2b3a0ad264472b96624b6d01b2cd5]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:49.607877  2977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4e6470e875614eec850a2fbafb26d548. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:49.609447  3103 tablet_bootstrap.cc:492] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088: Bootstrap starting.
I20240426 13:12:49.609825  3105 tablet_bootstrap.cc:492] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a: Bootstrap starting.
I20240426 13:12:49.610008  3104 tablet_bootstrap.cc:492] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a: Bootstrap starting.
I20240426 13:12:49.610302  3103 tablet_bootstrap.cc:654] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:49.610656  3105 tablet_bootstrap.cc:654] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:49.610736  3104 tablet_bootstrap.cc:654] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:49.611654  3104 tablet_bootstrap.cc:492] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a: No bootstrap required, opened a new log
I20240426 13:12:49.611749  3104 ts_tablet_manager.cc:1367] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:12:49.611781  3103 tablet_bootstrap.cc:492] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088: No bootstrap required, opened a new log
I20240426 13:12:49.611783  3105 tablet_bootstrap.cc:492] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a: No bootstrap required, opened a new log
I20240426 13:12:49.611891  3103 ts_tablet_manager.cc:1367] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088: Time spent bootstrapping tablet: real 0.002s	user 0.000s	sys 0.001s
I20240426 13:12:49.611891  3105 ts_tablet_manager.cc:1367] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:12:49.612149  3104 raft_consensus.cc:348] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.612285  3104 raft_consensus.cc:374] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:49.612329  3104 raft_consensus.cc:729] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cb611a3376940f3b1b7134f38b8662a, State: Initialized, Role: FOLLOWER
I20240426 13:12:49.612306  3103 raft_consensus.cc:348] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.612342  3105 raft_consensus.cc:348] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.612412  3103 raft_consensus.cc:374] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:49.612488  3105 raft_consensus.cc:374] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:49.612464  3104 consensus_queue.cc:260] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.612491  3103 raft_consensus.cc:729] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c3ce3c63b3d42fea7ce7a9b11d17088, State: Initialized, Role: FOLLOWER
I20240426 13:12:49.612526  3105 raft_consensus.cc:729] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d80b796768a4ae5a5ae41cf7e45997a, State: Initialized, Role: FOLLOWER
I20240426 13:12:49.612622  3103 consensus_queue.cc:260] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.612618  3105 consensus_queue.cc:260] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.612751  3104 ts_tablet_manager.cc:1398] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:49.612927  2937 heartbeater.cc:503] Master 127.0.99.190:40043 was elected leader, sending a full tablet report...
I20240426 13:12:49.612979  3103 ts_tablet_manager.cc:1398] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:12:49.613010  3013 heartbeater.cc:503] Master 127.0.99.190:40043 was elected leader, sending a full tablet report...
I20240426 13:12:49.612984  3105 ts_tablet_manager.cc:1398] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:49.613137  3088 heartbeater.cc:503] Master 127.0.99.190:40043 was elected leader, sending a full tablet report...
I20240426 13:12:49.636883  3109 raft_consensus.cc:482] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:12:49.636965  3109 raft_consensus.cc:504] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.637336  3109 leader_election.cc:288] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c3ce3c63b3d42fea7ce7a9b11d17088 (127.0.99.131:33491), 5d80b796768a4ae5a5ae41cf7e45997a (127.0.99.130:37275)
I20240426 13:12:49.640520  3062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e6470e875614eec850a2fbafb26d548" candidate_uuid: "5cb611a3376940f3b1b7134f38b8662a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" is_pre_election: true
I20240426 13:12:49.640645  3062 raft_consensus.cc:2454] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cb611a3376940f3b1b7134f38b8662a in term 0.
I20240426 13:12:49.640767  2987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e6470e875614eec850a2fbafb26d548" candidate_uuid: "5cb611a3376940f3b1b7134f38b8662a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" is_pre_election: true
I20240426 13:12:49.640890  2987 raft_consensus.cc:2454] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5cb611a3376940f3b1b7134f38b8662a in term 0.
I20240426 13:12:49.641000  2876 leader_election.cc:302] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c3ce3c63b3d42fea7ce7a9b11d17088, 5cb611a3376940f3b1b7134f38b8662a; no voters: 
I20240426 13:12:49.641142  3109 raft_consensus.cc:2781] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:12:49.641192  3109 raft_consensus.cc:482] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:12:49.641237  3109 raft_consensus.cc:3037] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:49.641903  3109 raft_consensus.cc:504] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.642221  3109 leader_election.cc:288] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [CANDIDATE]: Term 1 election: Requested vote from peers 1c3ce3c63b3d42fea7ce7a9b11d17088 (127.0.99.131:33491), 5d80b796768a4ae5a5ae41cf7e45997a (127.0.99.130:37275)
I20240426 13:12:49.642526  2987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e6470e875614eec850a2fbafb26d548" candidate_uuid: "5cb611a3376940f3b1b7134f38b8662a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d80b796768a4ae5a5ae41cf7e45997a"
I20240426 13:12:49.642623  2987 raft_consensus.cc:3037] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:49.642853  3062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4e6470e875614eec850a2fbafb26d548" candidate_uuid: "5cb611a3376940f3b1b7134f38b8662a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088"
I20240426 13:12:49.642949  3062 raft_consensus.cc:3037] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:49.643343  2987 raft_consensus.cc:2454] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cb611a3376940f3b1b7134f38b8662a in term 1.
I20240426 13:12:49.643591  2876 leader_election.cc:302] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cb611a3376940f3b1b7134f38b8662a, 5d80b796768a4ae5a5ae41cf7e45997a; no voters: 
I20240426 13:12:49.643646  3062 raft_consensus.cc:2454] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5cb611a3376940f3b1b7134f38b8662a in term 1.
I20240426 13:12:49.643766  3109 raft_consensus.cc:2781] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:49.643932  3109 raft_consensus.cc:686] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 1 LEADER]: Becoming Leader. State: Replica: 5cb611a3376940f3b1b7134f38b8662a, State: Running, Role: LEADER
I20240426 13:12:49.644071  3109 consensus_queue.cc:237] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } }
I20240426 13:12:49.645268  2808 catalog_manager.cc:5484] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5cb611a3376940f3b1b7134f38b8662a (127.0.99.129). New cstate: current_term: 1 leader_uuid: "5cb611a3376940f3b1b7134f38b8662a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cb611a3376940f3b1b7134f38b8662a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 43393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 } health_report { overall_health: UNKNOWN } } }
I20240426 13:12:49.662861  3062 raft_consensus.cc:1261] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 1 FOLLOWER]: Refusing update from remote peer 5cb611a3376940f3b1b7134f38b8662a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:49.662904  2987 raft_consensus.cc:1261] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 1 FOLLOWER]: Refusing update from remote peer 5cb611a3376940f3b1b7134f38b8662a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:12:49.663203  3109 consensus_queue.cc:1035] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c3ce3c63b3d42fea7ce7a9b11d17088" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 33491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:12:49.663311  3113 consensus_queue.cc:1035] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d80b796768a4ae5a5ae41cf7e45997a" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240426 13:12:59.676460  3120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Illegal state: Transaction 0 already open
W20240426 13:12:59.677211  3120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Illegal state: Cannot finalize transaction in state: OPEN
W20240426 13:12:59.680202  3120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Illegal state: Transaction 0 commit already in progress
W20240426 13:12:59.680896  3120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:12:59.684234  3120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Illegal state: Transaction 0 commit already in progress
W20240426 13:12:59.685111  3120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Illegal state: Transaction metadata for transaction 0 already set
W20240426 13:12:59.685998  3120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:12:59.686703  3120 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:47222: Illegal state: Transaction metadata for transaction 0 already exists
I20240426 13:12:59.687029   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:12:59.690042   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:59.690213   398 tablet_replica.cc:330] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a: stopping tablet replica
I20240426 13:12:59.690301   398 raft_consensus.cc:2229] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:59.690431   398 raft_consensus.cc:2258] T 4e6470e875614eec850a2fbafb26d548 P 5cb611a3376940f3b1b7134f38b8662a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:59.706560   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:12:59.707965   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:12:59.711067   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:59.711206   398 tablet_replica.cc:330] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a: stopping tablet replica
I20240426 13:12:59.711301   398 raft_consensus.cc:2229] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:59.711396   398 raft_consensus.cc:2258] T 4e6470e875614eec850a2fbafb26d548 P 5d80b796768a4ae5a5ae41cf7e45997a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:59.726637   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:12:59.728364   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:12:59.731719   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:12:59.731886   398 tablet_replica.cc:330] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088: stopping tablet replica
I20240426 13:12:59.731973   398 raft_consensus.cc:2229] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:12:59.732053   398 raft_consensus.cc:2258] T 4e6470e875614eec850a2fbafb26d548 P 1c3ce3c63b3d42fea7ce7a9b11d17088 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:59.747335   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:12:59.748689   398 master.cc:537] Master@127.0.99.190:40043 shutting down...
I20240426 13:12:59.752210   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:12:59.752301   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:12:59.752348   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P e10ce95ba70247a791dcd5130802b253: stopping tablet replica
I20240426 13:12:59.763995   398 master.cc:552] Master@127.0.99.190:40043 shutdown complete.
[       OK ] TxnParticipantITest.TestProxyIllegalStatesInCommitSequence (10337 ms)
[ RUN      ] TxnParticipantITest.TestProxyIllegalStatesInAbortSequence
I20240426 13:12:59.772272   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:45213
I20240426 13:12:59.772562   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:59.774951  3161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:59.775818  3162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:59.776648  3164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:59.777364   398 server_base.cc:1034] running on GCE node
I20240426 13:12:59.777484   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:59.777518   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:59.777530   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137179777531 us; error 0 us; skew 500 ppm
I20240426 13:12:59.778368   398 webserver.cc:460] Webserver started at http://127.0.99.190:39159/ using document root <none> and password file <none>
I20240426 13:12:59.778491   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:59.778530   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:59.778582   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:59.778954   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "d10d213f44224426bb025838edfdac2f"
format_stamp: "Formatted at 2024-04-26 13:12:59 on dist-test-slave-2jsl"
I20240426 13:12:59.780261   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:59.781060  3169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.781297   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:59.781358   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "d10d213f44224426bb025838edfdac2f"
format_stamp: "Formatted at 2024-04-26 13:12:59 on dist-test-slave-2jsl"
I20240426 13:12:59.781416   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:59.789855   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:59.790138   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:59.793258   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:45213
I20240426 13:12:59.794811  3221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:45213 every 8 connection(s)
I20240426 13:12:59.797562  3222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:59.799981  3222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f: Bootstrap starting.
I20240426 13:12:59.800556  3222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:59.801245  3222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f: No bootstrap required, opened a new log
I20240426 13:12:59.801492  3222 raft_consensus.cc:348] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d213f44224426bb025838edfdac2f" member_type: VOTER }
I20240426 13:12:59.801555  3222 raft_consensus.cc:374] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:59.801580  3222 raft_consensus.cc:729] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d10d213f44224426bb025838edfdac2f, State: Initialized, Role: FOLLOWER
I20240426 13:12:59.801661  3222 consensus_queue.cc:260] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d213f44224426bb025838edfdac2f" member_type: VOTER }
I20240426 13:12:59.801723  3222 raft_consensus.cc:388] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:12:59.801750  3222 raft_consensus.cc:482] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:12:59.801787  3222 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:59.802345  3222 raft_consensus.cc:504] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d213f44224426bb025838edfdac2f" member_type: VOTER }
I20240426 13:12:59.802434  3222 leader_election.cc:302] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d10d213f44224426bb025838edfdac2f; no voters: 
I20240426 13:12:59.802565  3222 leader_election.cc:288] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:12:59.802667  3225 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:59.802834  3225 raft_consensus.cc:686] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 1 LEADER]: Becoming Leader. State: Replica: d10d213f44224426bb025838edfdac2f, State: Running, Role: LEADER
I20240426 13:12:59.802912  3222 sys_catalog.cc:564] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:12:59.802942  3225 consensus_queue.cc:237] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d213f44224426bb025838edfdac2f" member_type: VOTER }
I20240426 13:12:59.803301  3227 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d10d213f44224426bb025838edfdac2f. Latest consensus state: current_term: 1 leader_uuid: "d10d213f44224426bb025838edfdac2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d213f44224426bb025838edfdac2f" member_type: VOTER } }
I20240426 13:12:59.803278  3226 sys_catalog.cc:455] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d10d213f44224426bb025838edfdac2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d10d213f44224426bb025838edfdac2f" member_type: VOTER } }
I20240426 13:12:59.803407  3227 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:59.803475  3226 sys_catalog.cc:458] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [sys.catalog]: This master's current role is: LEADER
I20240426 13:12:59.803989  3233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:12:59.804822  3233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:12:59.804986   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:12:59.806203  3233 catalog_manager.cc:1347] Generated new cluster ID: c556f4224aad4119bc49d206f7487f48
I20240426 13:12:59.806257  3233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:12:59.818915  3233 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:12:59.819355  3233 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:12:59.827167  3233 catalog_manager.cc:5857] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f: Generated new TSK 0
I20240426 13:12:59.827291  3233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:12:59.836863   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:59.839005  3243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:59.839898  3244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:59.840723  3246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:59.841480   398 server_base.cc:1034] running on GCE node
I20240426 13:12:59.841594   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:59.841622   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:59.841635   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137179841635 us; error 0 us; skew 500 ppm
I20240426 13:12:59.842370   398 webserver.cc:460] Webserver started at http://127.0.99.129:32887/ using document root <none> and password file <none>
I20240426 13:12:59.842492   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:59.842530   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:59.842576   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:59.842855   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "a02c521ac0334761b4594745dab04555"
format_stamp: "Formatted at 2024-04-26 13:12:59 on dist-test-slave-2jsl"
I20240426 13:12:59.844030   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:59.844717  3251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.844900   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:12:59.844949   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "a02c521ac0334761b4594745dab04555"
format_stamp: "Formatted at 2024-04-26 13:12:59 on dist-test-slave-2jsl"
I20240426 13:12:59.845005   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:59.851858   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:59.852095   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:59.852288   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:59.852658   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:59.852689   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.852725   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:59.852754   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.856181   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:42111
I20240426 13:12:59.856220  3314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:42111 every 8 connection(s)
I20240426 13:12:59.856783   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:59.858991  3319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:59.859727  3320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:59.860513  3322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:59.860643  3315 heartbeater.cc:347] Connected to a master server at 127.0.99.190:45213
I20240426 13:12:59.860714  3315 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:59.860890  3315 heartbeater.cc:511] Master 127.0.99.190:45213 requested a full tablet report, sending...
I20240426 13:12:59.861214   398 server_base.cc:1034] running on GCE node
I20240426 13:12:59.861352   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:59.861383   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:59.861402  3186 ts_manager.cc:194] Registered new tserver with Master: a02c521ac0334761b4594745dab04555 (127.0.99.129:42111)
I20240426 13:12:59.861403   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137179861402 us; error 0 us; skew 500 ppm
I20240426 13:12:59.862041  3186 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35866
I20240426 13:12:59.862277   398 webserver.cc:460] Webserver started at http://127.0.99.130:39453/ using document root <none> and password file <none>
I20240426 13:12:59.862407   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:59.862460   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:59.862519   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:59.862797   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b"
format_stamp: "Formatted at 2024-04-26 13:12:59 on dist-test-slave-2jsl"
I20240426 13:12:59.863929   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:12:59.864575  3327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.864739   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:12:59.864830   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b"
format_stamp: "Formatted at 2024-04-26 13:12:59 on dist-test-slave-2jsl"
I20240426 13:12:59.864886   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:59.879745   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:59.880198   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:59.880460   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:59.880820   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:59.880859   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.880898   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:59.880930   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.884635   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:38991
I20240426 13:12:59.884657  3390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:38991 every 8 connection(s)
I20240426 13:12:59.885285   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:12:59.887315  3394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:12:59.888098  3397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:59.888499  3391 heartbeater.cc:347] Connected to a master server at 127.0.99.190:45213
I20240426 13:12:59.888592  3391 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:59.888767  3391 heartbeater.cc:511] Master 127.0.99.190:45213 requested a full tablet report, sending...
I20240426 13:12:59.888962   398 server_base.cc:1034] running on GCE node
I20240426 13:12:59.889391  3186 ts_manager.cc:194] Registered new tserver with Master: 9cd5a5720a714b1aa2a74ff0cb504e4b (127.0.99.130:38991)
W20240426 13:12:59.889648  3395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:12:59.889858   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:12:59.889911   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:12:59.889926   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137179889926 us; error 0 us; skew 500 ppm
I20240426 13:12:59.890031  3186 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35882
I20240426 13:12:59.890740   398 webserver.cc:460] Webserver started at http://127.0.99.131:45095/ using document root <none> and password file <none>
I20240426 13:12:59.890877   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:12:59.890916   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:12:59.890980   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:12:59.891279   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "07710ac4847d4303b2c867d9528a9398"
format_stamp: "Formatted at 2024-04-26 13:12:59 on dist-test-slave-2jsl"
I20240426 13:12:59.892496   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:12:59.893321  3402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.893517   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:59.893571   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "07710ac4847d4303b2c867d9528a9398"
format_stamp: "Formatted at 2024-04-26 13:12:59 on dist-test-slave-2jsl"
I20240426 13:12:59.893636   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyIllegalStatesInAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:12:59.917831   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:12:59.918149   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:12:59.918391   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:12:59.918804   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:12:59.918841   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.918869   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:12:59.918902   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:12:59.922864   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:38753
I20240426 13:12:59.922930  3465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:38753 every 8 connection(s)
I20240426 13:12:59.926529  3466 heartbeater.cc:347] Connected to a master server at 127.0.99.190:45213
I20240426 13:12:59.926600  3466 heartbeater.cc:464] Registering TS with master...
I20240426 13:12:59.926748  3466 heartbeater.cc:511] Master 127.0.99.190:45213 requested a full tablet report, sending...
I20240426 13:12:59.927220  3186 ts_manager.cc:194] Registered new tserver with Master: 07710ac4847d4303b2c867d9528a9398 (127.0.99.131:38753)
I20240426 13:12:59.927654  3186 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35894
I20240426 13:12:59.927690   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004432612s
I20240426 13:12:59.928133   398 test_util.cc:268] Using random seed: -1447774653
I20240426 13:12:59.934230  3186 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:12:59.934571  3186 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:12:59.943095  3430 tablet_service.cc:1467] Processing CreateTablet for tablet 5f051f9d2b7d405c8d51e0ac834db904 (DEFAULT_TABLE table=test-workload [id=a0faa062a26d4e91b168e0305a631b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:59.943245  3279 tablet_service.cc:1467] Processing CreateTablet for tablet 5f051f9d2b7d405c8d51e0ac834db904 (DEFAULT_TABLE table=test-workload [id=a0faa062a26d4e91b168e0305a631b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:59.943343  3430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f051f9d2b7d405c8d51e0ac834db904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:59.943475  3279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f051f9d2b7d405c8d51e0ac834db904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:59.945340  3482 tablet_bootstrap.cc:492] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555: Bootstrap starting.
I20240426 13:12:59.945501  3355 tablet_service.cc:1467] Processing CreateTablet for tablet 5f051f9d2b7d405c8d51e0ac834db904 (DEFAULT_TABLE table=test-workload [id=a0faa062a26d4e91b168e0305a631b3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:12:59.945529  3481 tablet_bootstrap.cc:492] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398: Bootstrap starting.
I20240426 13:12:59.945724  3355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f051f9d2b7d405c8d51e0ac834db904. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:12:59.946326  3482 tablet_bootstrap.cc:654] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:59.946422  3481 tablet_bootstrap.cc:654] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:59.947419  3482 tablet_bootstrap.cc:492] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555: No bootstrap required, opened a new log
I20240426 13:12:59.947434  3481 tablet_bootstrap.cc:492] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398: No bootstrap required, opened a new log
I20240426 13:12:59.947582  3482 ts_tablet_manager.cc:1367] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:12:59.947618  3481 ts_tablet_manager.cc:1367] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:12:59.947932  3485 tablet_bootstrap.cc:492] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b: Bootstrap starting.
I20240426 13:12:59.947961  3481 raft_consensus.cc:348] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.948037  3481 raft_consensus.cc:374] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:59.948036  3482 raft_consensus.cc:348] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.948073  3481 raft_consensus.cc:729] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07710ac4847d4303b2c867d9528a9398, State: Initialized, Role: FOLLOWER
I20240426 13:12:59.948195  3482 raft_consensus.cc:374] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:59.948228  3482 raft_consensus.cc:729] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02c521ac0334761b4594745dab04555, State: Initialized, Role: FOLLOWER
I20240426 13:12:59.948297  3481 consensus_queue.cc:260] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.948380  3482 consensus_queue.cc:260] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.948529  3481 ts_tablet_manager.cc:1398] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:59.948678  3466 heartbeater.cc:503] Master 127.0.99.190:45213 was elected leader, sending a full tablet report...
I20240426 13:12:59.948748  3315 heartbeater.cc:503] Master 127.0.99.190:45213 was elected leader, sending a full tablet report...
I20240426 13:12:59.948688  3482 ts_tablet_manager.cc:1398] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:59.949247  3485 tablet_bootstrap.cc:654] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b: Neither blocks nor log segments found. Creating new log.
I20240426 13:12:59.950322  3485 tablet_bootstrap.cc:492] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b: No bootstrap required, opened a new log
I20240426 13:12:59.950402  3485 ts_tablet_manager.cc:1367] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:12:59.950783  3485 raft_consensus.cc:348] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.950882  3485 raft_consensus.cc:374] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:12:59.950913  3485 raft_consensus.cc:729] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd5a5720a714b1aa2a74ff0cb504e4b, State: Initialized, Role: FOLLOWER
I20240426 13:12:59.951023  3485 consensus_queue.cc:260] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.951326  3485 ts_tablet_manager.cc:1398] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:12:59.951462  3391 heartbeater.cc:503] Master 127.0.99.190:45213 was elected leader, sending a full tablet report...
I20240426 13:12:59.977460  3489 raft_consensus.cc:482] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:12:59.977542  3489 raft_consensus.cc:504] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.977993  3489 leader_election.cc:288] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07710ac4847d4303b2c867d9528a9398 (127.0.99.131:38753), a02c521ac0334761b4594745dab04555 (127.0.99.129:42111)
I20240426 13:12:59.981176  3440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f051f9d2b7d405c8d51e0ac834db904" candidate_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07710ac4847d4303b2c867d9528a9398" is_pre_election: true
I20240426 13:12:59.981217  3289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f051f9d2b7d405c8d51e0ac834db904" candidate_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a02c521ac0334761b4594745dab04555" is_pre_election: true
I20240426 13:12:59.981294  3440 raft_consensus.cc:2454] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cd5a5720a714b1aa2a74ff0cb504e4b in term 0.
I20240426 13:12:59.981320  3289 raft_consensus.cc:2454] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cd5a5720a714b1aa2a74ff0cb504e4b in term 0.
I20240426 13:12:59.981577  3331 leader_election.cc:302] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07710ac4847d4303b2c867d9528a9398, 9cd5a5720a714b1aa2a74ff0cb504e4b; no voters: 
I20240426 13:12:59.981724  3489 raft_consensus.cc:2781] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:12:59.981782  3489 raft_consensus.cc:482] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:12:59.981808  3489 raft_consensus.cc:3037] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:59.982558  3489 raft_consensus.cc:504] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.982864  3489 leader_election.cc:288] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [CANDIDATE]: Term 1 election: Requested vote from peers 07710ac4847d4303b2c867d9528a9398 (127.0.99.131:38753), a02c521ac0334761b4594745dab04555 (127.0.99.129:42111)
I20240426 13:12:59.983091  3289 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f051f9d2b7d405c8d51e0ac834db904" candidate_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a02c521ac0334761b4594745dab04555"
I20240426 13:12:59.983091  3440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f051f9d2b7d405c8d51e0ac834db904" candidate_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07710ac4847d4303b2c867d9528a9398"
I20240426 13:12:59.983209  3440 raft_consensus.cc:3037] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:59.983207  3289 raft_consensus.cc:3037] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:12:59.984028  3289 raft_consensus.cc:2454] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cd5a5720a714b1aa2a74ff0cb504e4b in term 1.
I20240426 13:12:59.984032  3440 raft_consensus.cc:2454] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cd5a5720a714b1aa2a74ff0cb504e4b in term 1.
I20240426 13:12:59.984270  3331 leader_election.cc:302] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07710ac4847d4303b2c867d9528a9398, 9cd5a5720a714b1aa2a74ff0cb504e4b; no voters: 
I20240426 13:12:59.984414  3489 raft_consensus.cc:2781] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:12:59.984613  3489 raft_consensus.cc:686] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 1 LEADER]: Becoming Leader. State: Replica: 9cd5a5720a714b1aa2a74ff0cb504e4b, State: Running, Role: LEADER
I20240426 13:12:59.984745  3489 consensus_queue.cc:237] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:12:59.986011  3186 catalog_manager.cc:5484] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cd5a5720a714b1aa2a74ff0cb504e4b (127.0.99.130). New cstate: current_term: 1 leader_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } health_report { overall_health: HEALTHY } } }
I20240426 13:13:00.017946  3289 raft_consensus.cc:1261] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 1 FOLLOWER]: Refusing update from remote peer 9cd5a5720a714b1aa2a74ff0cb504e4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:00.018121  3440 raft_consensus.cc:1261] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 1 FOLLOWER]: Refusing update from remote peer 9cd5a5720a714b1aa2a74ff0cb504e4b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:00.018440  3491 consensus_queue.cc:1035] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:00.018946  3491 consensus_queue.cc:1035] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [LEADER]: Connected to new peer: Peer: permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:10.544232  3491 raft_consensus.cc:979] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b: : Instructing follower a02c521ac0334761b4594745dab04555 to start an election
I20240426 13:13:10.544329  3491 raft_consensus.cc:1067] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 1 LEADER]: Signalling peer a02c521ac0334761b4594745dab04555 to start an election
I20240426 13:13:10.544680  3289 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5f051f9d2b7d405c8d51e0ac834db904"
dest_uuid: "a02c521ac0334761b4594745dab04555"
 from {username='slave'} at 127.0.0.1:58952
I20240426 13:13:10.544813  3289 raft_consensus.cc:482] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:13:10.544871  3289 raft_consensus.cc:3037] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:10.545629  3289 raft_consensus.cc:504] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:13:10.546025  3289 leader_election.cc:288] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [CANDIDATE]: Term 2 election: Requested vote from peers 07710ac4847d4303b2c867d9528a9398 (127.0.99.131:38753), 9cd5a5720a714b1aa2a74ff0cb504e4b (127.0.99.130:38991)
I20240426 13:13:10.549048  3365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f051f9d2b7d405c8d51e0ac834db904" candidate_uuid: "a02c521ac0334761b4594745dab04555" candidate_term: 2 candidate_status { last_received { term: 1 index: 3118 } } ignore_live_leader: true dest_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b"
I20240426 13:13:10.549103  3439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f051f9d2b7d405c8d51e0ac834db904" candidate_uuid: "a02c521ac0334761b4594745dab04555" candidate_term: 2 candidate_status { last_received { term: 1 index: 3118 } } ignore_live_leader: true dest_uuid: "07710ac4847d4303b2c867d9528a9398"
I20240426 13:13:10.549140  3365 raft_consensus.cc:3032] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:13:10.549203  3365 raft_consensus.cc:729] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9cd5a5720a714b1aa2a74ff0cb504e4b, State: Running, Role: LEADER
I20240426 13:13:10.549212  3439 raft_consensus.cc:3037] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:10.549346  3365 consensus_queue.cc:260] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3118, Committed index: 3118, Last appended: 1.3118, Last appended by leader: 3118, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:13:10.549482  3365 raft_consensus.cc:3037] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:10.550045  3439 raft_consensus.cc:2454] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02c521ac0334761b4594745dab04555 in term 2.
I20240426 13:13:10.550289  3365 raft_consensus.cc:2454] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02c521ac0334761b4594745dab04555 in term 2.
I20240426 13:13:10.550333  3255 leader_election.cc:302] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07710ac4847d4303b2c867d9528a9398, a02c521ac0334761b4594745dab04555; no voters: 
I20240426 13:13:10.550591  3544 raft_consensus.cc:2781] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:13:10.550756  3544 raft_consensus.cc:686] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 2 LEADER]: Becoming Leader. State: Replica: a02c521ac0334761b4594745dab04555, State: Running, Role: LEADER
I20240426 13:13:10.550896  3544 consensus_queue.cc:237] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3118, Committed index: 3118, Last appended: 1.3118, Last appended by leader: 3118, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } }
I20240426 13:13:10.552194  3177 catalog_manager.cc:5484] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 reported cstate change: term changed from 1 to 2, leader changed from 9cd5a5720a714b1aa2a74ff0cb504e4b (127.0.99.130) to a02c521ac0334761b4594745dab04555 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "a02c521ac0334761b4594745dab04555" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a02c521ac0334761b4594745dab04555" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 42111 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 } health_report { overall_health: UNKNOWN } } }
I20240426 13:13:10.557495  3365 raft_consensus.cc:1261] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 2 FOLLOWER]: Refusing update from remote peer a02c521ac0334761b4594745dab04555: Log matching property violated. Preceding OpId in replica: term: 1 index: 3118. Preceding OpId from leader: term: 2 index: 3120. (index mismatch)
I20240426 13:13:10.557734  3439 raft_consensus.cc:1261] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 2 FOLLOWER]: Refusing update from remote peer a02c521ac0334761b4594745dab04555: Log matching property violated. Preceding OpId in replica: term: 1 index: 3118. Preceding OpId from leader: term: 2 index: 3120. (index mismatch)
I20240426 13:13:10.557821  3545 consensus_queue.cc:1035] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cd5a5720a714b1aa2a74ff0cb504e4b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3119, Last known committed idx: 3118, Time since last communication: 0.000s
I20240426 13:13:10.558056  3544 consensus_queue.cc:1035] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07710ac4847d4303b2c867d9528a9398" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3119, Last known committed idx: 3118, Time since last communication: 0.000s
W20240426 13:13:10.561723  3552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39294: Illegal state: Transaction 0 already open
W20240426 13:13:10.562451  3552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39294: Illegal state: Cannot finalize transaction in state: OPEN
W20240426 13:13:10.565644  3552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39294: Illegal state: Transaction metadata for transaction 0 already set
W20240426 13:13:10.566315  3552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39294: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:13:10.566910  3552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39294: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:13:10.567516  3552 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:39294: Illegal state: Cannot begin committing transaction in state: INITIALIZING
I20240426 13:13:10.567797   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:13:10.570770   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:10.570906   398 tablet_replica.cc:330] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555: stopping tablet replica
I20240426 13:13:10.571044   398 raft_consensus.cc:2229] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:13:10.571172   398 raft_consensus.cc:2258] T 5f051f9d2b7d405c8d51e0ac834db904 P a02c521ac0334761b4594745dab04555 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:10.586663   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:13:10.588126   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:13:10.591365   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:10.591507   398 tablet_replica.cc:330] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b: stopping tablet replica
I20240426 13:13:10.591600   398 raft_consensus.cc:2229] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:10.591671   398 raft_consensus.cc:2258] T 5f051f9d2b7d405c8d51e0ac834db904 P 9cd5a5720a714b1aa2a74ff0cb504e4b [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:10.607537   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:13:10.609095   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:13:10.612092   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:10.612283   398 tablet_replica.cc:330] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398: stopping tablet replica
I20240426 13:13:10.612403   398 raft_consensus.cc:2229] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:10.612619   398 raft_consensus.cc:2258] T 5f051f9d2b7d405c8d51e0ac834db904 P 07710ac4847d4303b2c867d9528a9398 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:10.628276   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:13:10.629925   398 master.cc:537] Master@127.0.99.190:45213 shutting down...
I20240426 13:13:10.632953   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:13:10.633071   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:10.633117   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P d10d213f44224426bb025838edfdac2f: stopping tablet replica
I20240426 13:13:10.645152   398 master.cc:552] Master@127.0.99.190:45213 shutdown complete.
[       OK ] TxnParticipantITest.TestProxyIllegalStatesInAbortSequence (10881 ms)
[ RUN      ] TxnParticipantITest.TestProxyNonLeader
I20240426 13:13:10.654320   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:37057
I20240426 13:13:10.654601   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240426 13:13:10.657150   398 server_base.cc:1034] running on GCE node
W20240426 13:13:10.658027  3560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:10.658766  3561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:10.659502  3563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:10.659662   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:10.659698   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:10.659711   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137190659711 us; error 0 us; skew 500 ppm
I20240426 13:13:10.660576   398 webserver.cc:460] Webserver started at http://127.0.99.190:33201/ using document root <none> and password file <none>
I20240426 13:13:10.660714   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:10.660758   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:10.660828   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:10.661212   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "367e2f098e2b4ad0a4be5203e5aa652e"
format_stamp: "Formatted at 2024-04-26 13:13:10 on dist-test-slave-2jsl"
I20240426 13:13:10.662516   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:13:10.663221  3568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.663450   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:10.663501   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "367e2f098e2b4ad0a4be5203e5aa652e"
format_stamp: "Formatted at 2024-04-26 13:13:10 on dist-test-slave-2jsl"
I20240426 13:13:10.663549   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:10.677367   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:10.677623   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:10.681535   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:37057
I20240426 13:13:10.688264  3620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:37057 every 8 connection(s)
I20240426 13:13:10.688673  3621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:10.690443  3621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e: Bootstrap starting.
I20240426 13:13:10.691161  3621 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:10.692039  3621 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e: No bootstrap required, opened a new log
I20240426 13:13:10.692356  3621 raft_consensus.cc:348] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e2f098e2b4ad0a4be5203e5aa652e" member_type: VOTER }
I20240426 13:13:10.692435  3621 raft_consensus.cc:374] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:10.692471  3621 raft_consensus.cc:729] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 367e2f098e2b4ad0a4be5203e5aa652e, State: Initialized, Role: FOLLOWER
I20240426 13:13:10.692595  3621 consensus_queue.cc:260] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e2f098e2b4ad0a4be5203e5aa652e" member_type: VOTER }
I20240426 13:13:10.692654  3621 raft_consensus.cc:388] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:13:10.692698  3621 raft_consensus.cc:482] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:13:10.692747  3621 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:10.693331  3621 raft_consensus.cc:504] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e2f098e2b4ad0a4be5203e5aa652e" member_type: VOTER }
I20240426 13:13:10.693447  3621 leader_election.cc:302] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 367e2f098e2b4ad0a4be5203e5aa652e; no voters: 
I20240426 13:13:10.693620  3621 leader_election.cc:288] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:13:10.693696  3624 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:10.693864  3624 raft_consensus.cc:686] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 1 LEADER]: Becoming Leader. State: Replica: 367e2f098e2b4ad0a4be5203e5aa652e, State: Running, Role: LEADER
I20240426 13:13:10.694036  3621 sys_catalog.cc:564] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:13:10.694020  3624 consensus_queue.cc:237] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e2f098e2b4ad0a4be5203e5aa652e" member_type: VOTER }
I20240426 13:13:10.694442  3625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "367e2f098e2b4ad0a4be5203e5aa652e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e2f098e2b4ad0a4be5203e5aa652e" member_type: VOTER } }
I20240426 13:13:10.694468  3626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 367e2f098e2b4ad0a4be5203e5aa652e. Latest consensus state: current_term: 1 leader_uuid: "367e2f098e2b4ad0a4be5203e5aa652e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e2f098e2b4ad0a4be5203e5aa652e" member_type: VOTER } }
I20240426 13:13:10.694579  3625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:10.694638  3626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:10.695096  3629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:13:10.696007  3629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:13:10.696226   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:13:10.697580  3629 catalog_manager.cc:1347] Generated new cluster ID: 7c975d8d352d4e82ac840666ca1f5663
I20240426 13:13:10.697631  3629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:13:10.702956  3629 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:13:10.703459  3629 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:13:10.708858  3629 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e: Generated new TSK 0
I20240426 13:13:10.708986  3629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:13:10.712155   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:10.714584  3642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:10.715502  3645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:10.716246  3643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:10.716959   398 server_base.cc:1034] running on GCE node
I20240426 13:13:10.717061   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:10.717089   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:10.717101   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137190717102 us; error 0 us; skew 500 ppm
I20240426 13:13:10.717818   398 webserver.cc:460] Webserver started at http://127.0.99.129:39453/ using document root <none> and password file <none>
I20240426 13:13:10.717927   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:10.717959   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:10.718047   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:10.718346   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "b625b4faed124910b0609cff7beb66f3"
format_stamp: "Formatted at 2024-04-26 13:13:10 on dist-test-slave-2jsl"
I20240426 13:13:10.719561   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:10.720343  3650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.720553   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:10.720609   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "b625b4faed124910b0609cff7beb66f3"
format_stamp: "Formatted at 2024-04-26 13:13:10 on dist-test-slave-2jsl"
I20240426 13:13:10.720674   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:10.746640   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:10.746955   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:10.747186   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:10.747566   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:10.747602   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.747644   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:10.747678   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.751164   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:33017
I20240426 13:13:10.751188  3713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:33017 every 8 connection(s)
I20240426 13:13:10.751806   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:10.754348  3718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:10.755208  3719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:10.755645  3714 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37057
I20240426 13:13:10.755753  3714 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:10.755959  3714 heartbeater.cc:511] Master 127.0.99.190:37057 requested a full tablet report, sending...
W20240426 13:13:10.756019  3721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:10.756559  3585 ts_manager.cc:194] Registered new tserver with Master: b625b4faed124910b0609cff7beb66f3 (127.0.99.129:33017)
I20240426 13:13:10.756799   398 server_base.cc:1034] running on GCE node
I20240426 13:13:10.756951   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:10.756985   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:10.756999   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137190756999 us; error 0 us; skew 500 ppm
I20240426 13:13:10.757464  3585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37500
I20240426 13:13:10.757944   398 webserver.cc:460] Webserver started at http://127.0.99.130:44647/ using document root <none> and password file <none>
I20240426 13:13:10.758105   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:10.758164   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:10.758258   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:10.758615   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "73e653816aa54a91900f59e5885cef07"
format_stamp: "Formatted at 2024-04-26 13:13:10 on dist-test-slave-2jsl"
I20240426 13:13:10.759825   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:13:10.760699  3726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.760907   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:10.760970   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "73e653816aa54a91900f59e5885cef07"
format_stamp: "Formatted at 2024-04-26 13:13:10 on dist-test-slave-2jsl"
I20240426 13:13:10.761036   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:10.772433   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:10.772682   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:10.772903   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:10.773254   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:10.773286   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.773325   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:10.773352   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.776664   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:42691
I20240426 13:13:10.776705  3789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:42691 every 8 connection(s)
I20240426 13:13:10.777248   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:10.779582  3794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:10.780416  3793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:10.780575  3790 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37057
I20240426 13:13:10.780642  3790 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:10.780778  3790 heartbeater.cc:511] Master 127.0.99.190:37057 requested a full tablet report, sending...
I20240426 13:13:10.781239   398 server_base.cc:1034] running on GCE node
I20240426 13:13:10.781263  3585 ts_manager.cc:194] Registered new tserver with Master: 73e653816aa54a91900f59e5885cef07 (127.0.99.130:42691)
I20240426 13:13:10.781756  3585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37508
W20240426 13:13:10.781999  3796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:10.782195   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:10.782241   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:10.782263   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137190782262 us; error 0 us; skew 500 ppm
I20240426 13:13:10.782964   398 webserver.cc:460] Webserver started at http://127.0.99.131:45571/ using document root <none> and password file <none>
I20240426 13:13:10.783064   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:10.783098   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:10.783152   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:10.783464   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "9aace8e0e5424eceba851673b3edd98d"
format_stamp: "Formatted at 2024-04-26 13:13:10 on dist-test-slave-2jsl"
I20240426 13:13:10.784577   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:10.785322  3801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.785584   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:10.785635   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "9aace8e0e5424eceba851673b3edd98d"
format_stamp: "Formatted at 2024-04-26 13:13:10 on dist-test-slave-2jsl"
I20240426 13:13:10.785679   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyNonLeader.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:10.792909   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:10.793124   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:10.793313   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:10.793659   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:10.793692   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.793718   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:10.793751   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:10.797811   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:46203
I20240426 13:13:10.797857  3864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:46203 every 8 connection(s)
I20240426 13:13:10.801477  3865 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37057
I20240426 13:13:10.801581  3865 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:10.801716  3865 heartbeater.cc:511] Master 127.0.99.190:37057 requested a full tablet report, sending...
I20240426 13:13:10.802261  3585 ts_manager.cc:194] Registered new tserver with Master: 9aace8e0e5424eceba851673b3edd98d (127.0.99.131:46203)
I20240426 13:13:10.802649   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00447095s
I20240426 13:13:10.802682  3585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37512
I20240426 13:13:10.803078   398 test_util.cc:268] Using random seed: -1436899707
I20240426 13:13:10.808630  3585 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37514:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:13:10.808974  3585 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:13:10.816710  3678 tablet_service.cc:1467] Processing CreateTablet for tablet 2fbaa55382c6409fa044e294771ceb73 (DEFAULT_TABLE table=test-workload [id=699c255ccca44e7cb76272e9aa4393d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:10.816951  3678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fbaa55382c6409fa044e294771ceb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:10.817478  3829 tablet_service.cc:1467] Processing CreateTablet for tablet 2fbaa55382c6409fa044e294771ceb73 (DEFAULT_TABLE table=test-workload [id=699c255ccca44e7cb76272e9aa4393d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:10.817669  3829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fbaa55382c6409fa044e294771ceb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:10.818744  3880 tablet_bootstrap.cc:492] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: Bootstrap starting.
I20240426 13:13:10.818857  3754 tablet_service.cc:1467] Processing CreateTablet for tablet 2fbaa55382c6409fa044e294771ceb73 (DEFAULT_TABLE table=test-workload [id=699c255ccca44e7cb76272e9aa4393d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:10.819061  3754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2fbaa55382c6409fa044e294771ceb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:10.819490  3881 tablet_bootstrap.cc:492] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d: Bootstrap starting.
I20240426 13:13:10.819991  3880 tablet_bootstrap.cc:654] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:10.820400  3881 tablet_bootstrap.cc:654] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:10.820742  3883 tablet_bootstrap.cc:492] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07: Bootstrap starting.
I20240426 13:13:10.821218  3880 tablet_bootstrap.cc:492] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: No bootstrap required, opened a new log
I20240426 13:13:10.821281  3880 ts_tablet_manager.cc:1367] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:13:10.821599  3880 raft_consensus.cc:348] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:10.821642  3883 tablet_bootstrap.cc:654] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:10.821678  3880 raft_consensus.cc:374] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:10.821699  3880 raft_consensus.cc:729] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b625b4faed124910b0609cff7beb66f3, State: Initialized, Role: FOLLOWER
I20240426 13:13:10.821799  3880 consensus_queue.cc:260] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:10.822173  3880 ts_tablet_manager.cc:1398] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:10.822180  3714 heartbeater.cc:503] Master 127.0.99.190:37057 was elected leader, sending a full tablet report...
I20240426 13:13:10.822788  3881 tablet_bootstrap.cc:492] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d: No bootstrap required, opened a new log
I20240426 13:13:10.822866  3881 ts_tablet_manager.cc:1367] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:13:10.822906  3883 tablet_bootstrap.cc:492] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07: No bootstrap required, opened a new log
I20240426 13:13:10.822968  3883 ts_tablet_manager.cc:1367] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:13:10.823230  3881 raft_consensus.cc:348] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:10.823338  3881 raft_consensus.cc:374] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:10.823383  3881 raft_consensus.cc:729] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aace8e0e5424eceba851673b3edd98d, State: Initialized, Role: FOLLOWER
I20240426 13:13:10.823495  3881 consensus_queue.cc:260] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:10.823851  3881 ts_tablet_manager.cc:1398] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:10.824018  3865 heartbeater.cc:503] Master 127.0.99.190:37057 was elected leader, sending a full tablet report...
I20240426 13:13:10.823266  3883 raft_consensus.cc:348] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:10.824288  3883 raft_consensus.cc:374] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:10.824352  3883 raft_consensus.cc:729] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73e653816aa54a91900f59e5885cef07, State: Initialized, Role: FOLLOWER
I20240426 13:13:10.824470  3883 consensus_queue.cc:260] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:10.824819  3883 ts_tablet_manager.cc:1398] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:13:10.824971  3790 heartbeater.cc:503] Master 127.0.99.190:37057 was elected leader, sending a full tablet report...
W20240426 13:13:11.001739  3715 tablet.cc:2369] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 13:13:11.072619  3888 raft_consensus.cc:482] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:13:11.072749  3888 raft_consensus.cc:504] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:11.073173  3888 leader_election.cc:288] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b625b4faed124910b0609cff7beb66f3 (127.0.99.129:33017), 9aace8e0e5424eceba851673b3edd98d (127.0.99.131:46203)
I20240426 13:13:11.076704  3688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fbaa55382c6409fa044e294771ceb73" candidate_uuid: "73e653816aa54a91900f59e5885cef07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625b4faed124910b0609cff7beb66f3" is_pre_election: true
I20240426 13:13:11.076807  3839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fbaa55382c6409fa044e294771ceb73" candidate_uuid: "73e653816aa54a91900f59e5885cef07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aace8e0e5424eceba851673b3edd98d" is_pre_election: true
I20240426 13:13:11.076810  3688 raft_consensus.cc:2454] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73e653816aa54a91900f59e5885cef07 in term 0.
I20240426 13:13:11.076989  3839 raft_consensus.cc:2454] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73e653816aa54a91900f59e5885cef07 in term 0.
I20240426 13:13:11.077190  3729 leader_election.cc:302] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73e653816aa54a91900f59e5885cef07, b625b4faed124910b0609cff7beb66f3; no voters: 
I20240426 13:13:11.077337  3888 raft_consensus.cc:2781] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:13:11.077391  3888 raft_consensus.cc:482] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:13:11.077440  3888 raft_consensus.cc:3037] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:11.078182  3888 raft_consensus.cc:504] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:11.078452  3888 leader_election.cc:288] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [CANDIDATE]: Term 1 election: Requested vote from peers b625b4faed124910b0609cff7beb66f3 (127.0.99.129:33017), 9aace8e0e5424eceba851673b3edd98d (127.0.99.131:46203)
I20240426 13:13:11.078711  3688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fbaa55382c6409fa044e294771ceb73" candidate_uuid: "73e653816aa54a91900f59e5885cef07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625b4faed124910b0609cff7beb66f3"
I20240426 13:13:11.078748  3839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fbaa55382c6409fa044e294771ceb73" candidate_uuid: "73e653816aa54a91900f59e5885cef07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aace8e0e5424eceba851673b3edd98d"
I20240426 13:13:11.078787  3688 raft_consensus.cc:3037] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:11.078856  3839 raft_consensus.cc:3037] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:11.079593  3688 raft_consensus.cc:2454] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73e653816aa54a91900f59e5885cef07 in term 1.
I20240426 13:13:11.079649  3839 raft_consensus.cc:2454] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73e653816aa54a91900f59e5885cef07 in term 1.
I20240426 13:13:11.079839  3729 leader_election.cc:302] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73e653816aa54a91900f59e5885cef07, b625b4faed124910b0609cff7beb66f3; no voters: 
I20240426 13:13:11.079995  3888 raft_consensus.cc:2781] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:11.080188  3888 raft_consensus.cc:686] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 1 LEADER]: Becoming Leader. State: Replica: 73e653816aa54a91900f59e5885cef07, State: Running, Role: LEADER
I20240426 13:13:11.080508  3888 consensus_queue.cc:237] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:11.081869  3585 catalog_manager.cc:5484] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 73e653816aa54a91900f59e5885cef07 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "73e653816aa54a91900f59e5885cef07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } health_report { overall_health: HEALTHY } } }
I20240426 13:13:11.120060  3839 raft_consensus.cc:1261] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 1 FOLLOWER]: Refusing update from remote peer 73e653816aa54a91900f59e5885cef07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:11.120111  3688 raft_consensus.cc:1261] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 1 FOLLOWER]: Refusing update from remote peer 73e653816aa54a91900f59e5885cef07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:11.120568  3890 consensus_queue.cc:1035] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:11.120893  3890 consensus_queue.cc:1035] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:11.125561  3897 mvcc.cc:205] Tried to move back new op lower bound from 7021105934828896256 to 7021105934667128832. Current Snapshot: MvccSnapshot[applied={T|T < 7021105934828896256}]
I20240426 13:13:21.674412  3914 raft_consensus.cc:979] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07: : Instructing follower b625b4faed124910b0609cff7beb66f3 to start an election
I20240426 13:13:21.674511  3914 raft_consensus.cc:1067] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 1 LEADER]: Signalling peer b625b4faed124910b0609cff7beb66f3 to start an election
I20240426 13:13:21.674872  3688 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2fbaa55382c6409fa044e294771ceb73"
dest_uuid: "b625b4faed124910b0609cff7beb66f3"
 from {username='slave'} at 127.0.0.1:36274
I20240426 13:13:21.674993  3688 raft_consensus.cc:482] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:13:21.675048  3688 raft_consensus.cc:3037] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:21.675895  3688 raft_consensus.cc:504] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:21.676311  3688 leader_election.cc:288] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [CANDIDATE]: Term 2 election: Requested vote from peers 9aace8e0e5424eceba851673b3edd98d (127.0.99.131:46203), 73e653816aa54a91900f59e5885cef07 (127.0.99.130:42691)
I20240426 13:13:21.679961  3764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fbaa55382c6409fa044e294771ceb73" candidate_uuid: "b625b4faed124910b0609cff7beb66f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3060 } } ignore_live_leader: true dest_uuid: "73e653816aa54a91900f59e5885cef07"
I20240426 13:13:21.679961  3839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2fbaa55382c6409fa044e294771ceb73" candidate_uuid: "b625b4faed124910b0609cff7beb66f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3060 } } ignore_live_leader: true dest_uuid: "9aace8e0e5424eceba851673b3edd98d"
I20240426 13:13:21.680102  3764 raft_consensus.cc:3032] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:13:21.680123  3839 raft_consensus.cc:3037] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:21.680166  3764 raft_consensus.cc:729] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73e653816aa54a91900f59e5885cef07, State: Running, Role: LEADER
I20240426 13:13:21.680328  3764 consensus_queue.cc:260] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3060, Committed index: 3060, Last appended: 1.3060, Last appended by leader: 3060, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:21.680495  3764 raft_consensus.cc:3037] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:21.681118  3839 raft_consensus.cc:2454] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b625b4faed124910b0609cff7beb66f3 in term 2.
I20240426 13:13:21.681418  3653 leader_election.cc:302] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aace8e0e5424eceba851673b3edd98d, b625b4faed124910b0609cff7beb66f3; no voters: 
I20240426 13:13:21.681418  3764 raft_consensus.cc:2454] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b625b4faed124910b0609cff7beb66f3 in term 2.
I20240426 13:13:21.681738  3945 raft_consensus.cc:2781] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:13:21.681922  3945 raft_consensus.cc:686] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 2 LEADER]: Becoming Leader. State: Replica: b625b4faed124910b0609cff7beb66f3, State: Running, Role: LEADER
I20240426 13:13:21.682106  3945 consensus_queue.cc:237] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3060, Committed index: 3060, Last appended: 1.3060, Last appended by leader: 3060, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } }
I20240426 13:13:21.683364  3582 catalog_manager.cc:5484] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 reported cstate change: term changed from 1 to 2, leader changed from 73e653816aa54a91900f59e5885cef07 (127.0.99.130) to b625b4faed124910b0609cff7beb66f3 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "b625b4faed124910b0609cff7beb66f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625b4faed124910b0609cff7beb66f3" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aace8e0e5424eceba851673b3edd98d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73e653816aa54a91900f59e5885cef07" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42691 } health_report { overall_health: UNKNOWN } } }
W20240426 13:13:21.692739  3754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:35960: Illegal state: replica 73e653816aa54a91900f59e5885cef07 is not leader of this config: current role FOLLOWER
W20240426 13:13:21.693379  3754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:35960: Illegal state: replica 73e653816aa54a91900f59e5885cef07 is not leader of this config: current role FOLLOWER
W20240426 13:13:21.694028  3754 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:35960: Illegal state: replica 73e653816aa54a91900f59e5885cef07 is not leader of this config: current role FOLLOWER
I20240426 13:13:21.694358   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:13:21.698015   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:21.698155   398 tablet_replica.cc:330] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: stopping tablet replica
I20240426 13:13:21.698254   398 raft_consensus.cc:2229] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:13:21.698385   398 pending_rounds.cc:62] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: Trying to abort 1 pending ops.
I20240426 13:13:21.698442   398 pending_rounds.cc:68] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3: Aborting op as it isn't in flight: id { term: 2 index: 3061 } timestamp: 7021105978091335680 op_type: NO_OP noop_request { }
I20240426 13:13:21.698508   398 raft_consensus.cc:2866] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240426 13:13:21.698565   398 raft_consensus.cc:2258] T 2fbaa55382c6409fa044e294771ceb73 P b625b4faed124910b0609cff7beb66f3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:21.715157   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:13:21.716722   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:13:21.720860   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:21.721081   398 tablet_replica.cc:330] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07: stopping tablet replica
I20240426 13:13:21.721186   398 raft_consensus.cc:2229] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:21.721279   398 raft_consensus.cc:2258] T 2fbaa55382c6409fa044e294771ceb73 P 73e653816aa54a91900f59e5885cef07 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:21.736855   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:13:21.738330   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:13:21.742877   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:21.743038   398 tablet_replica.cc:330] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d: stopping tablet replica
I20240426 13:13:21.743114   398 raft_consensus.cc:2229] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:21.743252   398 raft_consensus.cc:2258] T 2fbaa55382c6409fa044e294771ceb73 P 9aace8e0e5424eceba851673b3edd98d [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:21.758292   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:13:21.759692   398 master.cc:537] Master@127.0.99.190:37057 shutting down...
I20240426 13:13:21.762943   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:13:21.763047   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:21.763100   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 367e2f098e2b4ad0a4be5203e5aa652e: stopping tablet replica
I20240426 13:13:21.775027   398 master.cc:552] Master@127.0.99.190:37057 shutdown complete.
[       OK ] TxnParticipantITest.TestProxyNonLeader (11129 ms)
[ RUN      ] TxnParticipantITest.TestProxyTabletBootstrapping
I20240426 13:13:21.783247   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:35873
I20240426 13:13:21.783524   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:21.785881  3953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:21.786725  3952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:21.787520  3955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:21.788429   398 server_base.cc:1034] running on GCE node
I20240426 13:13:21.788552   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:21.788583   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:21.788595   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137201788595 us; error 0 us; skew 500 ppm
I20240426 13:13:21.789299   398 webserver.cc:460] Webserver started at http://127.0.99.190:45047/ using document root <none> and password file <none>
I20240426 13:13:21.789410   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:21.789448   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:21.789505   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:21.789803   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "b506b67485bd48659f1fbc5db7e11a1a"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:21.791117   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:21.791903  3960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.792125   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:21.792192   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "b506b67485bd48659f1fbc5db7e11a1a"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:21.792256   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:21.815372   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:21.815610   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:21.818883   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:35873
I20240426 13:13:21.824766  4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:21.824792  4012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:35873 every 8 connection(s)
I20240426 13:13:21.826265  4013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a: Bootstrap starting.
I20240426 13:13:21.826834  4013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:21.827581  4013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a: No bootstrap required, opened a new log
I20240426 13:13:21.827855  4013 raft_consensus.cc:348] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b506b67485bd48659f1fbc5db7e11a1a" member_type: VOTER }
I20240426 13:13:21.827924  4013 raft_consensus.cc:374] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:21.827952  4013 raft_consensus.cc:729] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b506b67485bd48659f1fbc5db7e11a1a, State: Initialized, Role: FOLLOWER
I20240426 13:13:21.828074  4013 consensus_queue.cc:260] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b506b67485bd48659f1fbc5db7e11a1a" member_type: VOTER }
I20240426 13:13:21.828131  4013 raft_consensus.cc:388] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:13:21.828162  4013 raft_consensus.cc:482] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:13:21.828203  4013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:21.828768  4013 raft_consensus.cc:504] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b506b67485bd48659f1fbc5db7e11a1a" member_type: VOTER }
I20240426 13:13:21.828871  4013 leader_election.cc:302] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b506b67485bd48659f1fbc5db7e11a1a; no voters: 
I20240426 13:13:21.829033  4013 leader_election.cc:288] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:13:21.829118  4016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:21.829262  4016 raft_consensus.cc:686] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 1 LEADER]: Becoming Leader. State: Replica: b506b67485bd48659f1fbc5db7e11a1a, State: Running, Role: LEADER
I20240426 13:13:21.829416  4013 sys_catalog.cc:564] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:13:21.829423  4016 consensus_queue.cc:237] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b506b67485bd48659f1fbc5db7e11a1a" member_type: VOTER }
I20240426 13:13:21.829833  4017 sys_catalog.cc:455] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b506b67485bd48659f1fbc5db7e11a1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b506b67485bd48659f1fbc5db7e11a1a" member_type: VOTER } }
I20240426 13:13:21.829871  4018 sys_catalog.cc:455] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b506b67485bd48659f1fbc5db7e11a1a. Latest consensus state: current_term: 1 leader_uuid: "b506b67485bd48659f1fbc5db7e11a1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b506b67485bd48659f1fbc5db7e11a1a" member_type: VOTER } }
I20240426 13:13:21.830039  4018 sys_catalog.cc:458] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:21.830021  4017 sys_catalog.cc:458] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:21.830546  4023 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:13:21.831192  4023 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:13:21.831365   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:13:21.832808  4023 catalog_manager.cc:1347] Generated new cluster ID: 359365df6f62402899012a0583f44813
I20240426 13:13:21.832855  4023 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:13:21.837872  4023 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:13:21.838392  4023 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:13:21.843834  4023 catalog_manager.cc:5857] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a: Generated new TSK 0
I20240426 13:13:21.843950  4023 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:13:21.847203   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:21.849478  4035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:21.850289  4034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:21.851136  4037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:21.851920   398 server_base.cc:1034] running on GCE node
I20240426 13:13:21.852046   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:21.852095   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:21.852109   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137201852109 us; error 0 us; skew 500 ppm
I20240426 13:13:21.852834   398 webserver.cc:460] Webserver started at http://127.0.99.129:44999/ using document root <none> and password file <none>
I20240426 13:13:21.852948   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:21.852986   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:21.853056   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:21.853356   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "49e48ccc5c534611a13c30a21898bb06"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:21.854599   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:13:21.855322  4042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.855525   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:13:21.855576   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "49e48ccc5c534611a13c30a21898bb06"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:21.855629   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:21.872068   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:21.872355   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:21.872550   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:21.872891   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:21.872926   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.872962   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:21.872992   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.876503   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:40519
I20240426 13:13:21.876528  4105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:40519 every 8 connection(s)
I20240426 13:13:21.877184   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:21.879384  4110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:21.880169  4111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:21.881042  4106 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35873
W20240426 13:13:21.881090  4113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:21.881125  4106 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:21.881291  4106 heartbeater.cc:511] Master 127.0.99.190:35873 requested a full tablet report, sending...
I20240426 13:13:21.881810   398 server_base.cc:1034] running on GCE node
I20240426 13:13:21.881852  3977 ts_manager.cc:194] Registered new tserver with Master: 49e48ccc5c534611a13c30a21898bb06 (127.0.99.129:40519)
I20240426 13:13:21.882021   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:21.882066   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:21.882086   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137201882086 us; error 0 us; skew 500 ppm
I20240426 13:13:21.882781  3977 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57116
I20240426 13:13:21.882954   398 webserver.cc:460] Webserver started at http://127.0.99.130:33059/ using document root <none> and password file <none>
I20240426 13:13:21.883087   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:21.883136   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:21.883208   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:21.883540   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "0d9f076426a441f8b396f442cdb98f5b"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:21.884721   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:13:21.885475  4118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.885665   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:13:21.885725   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "0d9f076426a441f8b396f442cdb98f5b"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:21.885782   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:21.892270   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:21.892516   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:21.892740   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:21.893124   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:21.893162   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.893208   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:21.893235   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.897090   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:37089
I20240426 13:13:21.897154  4181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:37089 every 8 connection(s)
I20240426 13:13:21.897735   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:21.900223  4185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:21.900876  4186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:21.901432  4182 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35873
I20240426 13:13:21.901504  4182 heartbeater.cc:464] Registering TS with master...
W20240426 13:13:21.901712  4188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:21.901736  4182 heartbeater.cc:511] Master 127.0.99.190:35873 requested a full tablet report, sending...
I20240426 13:13:21.902441  3977 ts_manager.cc:194] Registered new tserver with Master: 0d9f076426a441f8b396f442cdb98f5b (127.0.99.130:37089)
I20240426 13:13:21.902665   398 server_base.cc:1034] running on GCE node
I20240426 13:13:21.902853   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:21.902896   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:21.902915   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137201902915 us; error 0 us; skew 500 ppm
I20240426 13:13:21.903100  3977 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57118
I20240426 13:13:21.903842   398 webserver.cc:460] Webserver started at http://127.0.99.131:36687/ using document root <none> and password file <none>
I20240426 13:13:21.903980   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:21.904026   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:21.904096   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:21.904440   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "1149982f4cff4aa4ad64f110d25d5eed"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:21.905800   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:21.906652  4193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.906847   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:21.906908   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "1149982f4cff4aa4ad64f110d25d5eed"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:21.906984   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:21.929145   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:21.929417   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:21.929682   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:21.930131   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:21.930168   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.930209   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:21.930239   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:21.934248   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:41967
I20240426 13:13:21.934299  4256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:41967 every 8 connection(s)
I20240426 13:13:21.938040  4257 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35873
I20240426 13:13:21.938112  4257 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:21.938290  4257 heartbeater.cc:511] Master 127.0.99.190:35873 requested a full tablet report, sending...
I20240426 13:13:21.938771  3977 ts_manager.cc:194] Registered new tserver with Master: 1149982f4cff4aa4ad64f110d25d5eed (127.0.99.131:41967)
I20240426 13:13:21.939114   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004515028s
I20240426 13:13:21.939282  3977 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57126
I20240426 13:13:21.939603   398 test_util.cc:268] Using random seed: -1425763183
I20240426 13:13:21.945567  3977 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57136:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:13:21.945894  3977 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:13:21.954275  4221 tablet_service.cc:1467] Processing CreateTablet for tablet d393e2bc65b24049a6542894a3a6a1d9 (DEFAULT_TABLE table=test-workload [id=00d58521033f4bd8941f63098d1a9695]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:21.954350  4146 tablet_service.cc:1467] Processing CreateTablet for tablet d393e2bc65b24049a6542894a3a6a1d9 (DEFAULT_TABLE table=test-workload [id=00d58521033f4bd8941f63098d1a9695]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:21.954512  4146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d393e2bc65b24049a6542894a3a6a1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:21.954512  4221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d393e2bc65b24049a6542894a3a6a1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:21.954672  4070 tablet_service.cc:1467] Processing CreateTablet for tablet d393e2bc65b24049a6542894a3a6a1d9 (DEFAULT_TABLE table=test-workload [id=00d58521033f4bd8941f63098d1a9695]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:21.954854  4070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d393e2bc65b24049a6542894a3a6a1d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:21.956403  4272 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed: Bootstrap starting.
I20240426 13:13:21.956782  4274 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b: Bootstrap starting.
I20240426 13:13:21.957432  4274 tablet_bootstrap.cc:654] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:21.957438  4272 tablet_bootstrap.cc:654] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:21.957943  4273 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Bootstrap starting.
I20240426 13:13:21.958397  4274 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b: No bootstrap required, opened a new log
I20240426 13:13:21.958397  4272 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed: No bootstrap required, opened a new log
I20240426 13:13:21.958532  4274 ts_tablet_manager.cc:1367] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:13:21.958585  4272 ts_tablet_manager.cc:1367] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:13:21.958653  4273 tablet_bootstrap.cc:654] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:21.958866  4274 raft_consensus.cc:348] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:21.958945  4274 raft_consensus.cc:374] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:21.958986  4274 raft_consensus.cc:729] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d9f076426a441f8b396f442cdb98f5b, State: Initialized, Role: FOLLOWER
I20240426 13:13:21.958976  4272 raft_consensus.cc:348] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:21.959156  4272 raft_consensus.cc:374] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:21.959136  4274 consensus_queue.cc:260] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:21.959223  4272 raft_consensus.cc:729] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1149982f4cff4aa4ad64f110d25d5eed, State: Initialized, Role: FOLLOWER
I20240426 13:13:21.959374  4272 consensus_queue.cc:260] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:21.959489  4182 heartbeater.cc:503] Master 127.0.99.190:35873 was elected leader, sending a full tablet report...
I20240426 13:13:21.959658  4273 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: No bootstrap required, opened a new log
I20240426 13:13:21.959722  4272 ts_tablet_manager.cc:1398] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:21.959722  4273 ts_tablet_manager.cc:1367] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Time spent bootstrapping tablet: real 0.002s	user 0.000s	sys 0.001s
I20240426 13:13:21.959877  4257 heartbeater.cc:503] Master 127.0.99.190:35873 was elected leader, sending a full tablet report...
I20240426 13:13:21.960124  4273 raft_consensus.cc:348] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:21.960237  4273 raft_consensus.cc:374] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:21.960265  4273 raft_consensus.cc:729] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e48ccc5c534611a13c30a21898bb06, State: Initialized, Role: FOLLOWER
I20240426 13:13:21.960350  4273 consensus_queue.cc:260] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:21.960608  4273 ts_tablet_manager.cc:1398] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:21.960749  4106 heartbeater.cc:503] Master 127.0.99.190:35873 was elected leader, sending a full tablet report...
I20240426 13:13:21.959489  4274 ts_tablet_manager.cc:1398] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:22.272166  4278 raft_consensus.cc:482] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:13:22.272330  4278 raft_consensus.cc:504] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:22.272876  4278 leader_election.cc:288] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49e48ccc5c534611a13c30a21898bb06 (127.0.99.129:40519), 1149982f4cff4aa4ad64f110d25d5eed (127.0.99.131:41967)
I20240426 13:13:22.276405  4080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393e2bc65b24049a6542894a3a6a1d9" candidate_uuid: "0d9f076426a441f8b396f442cdb98f5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e48ccc5c534611a13c30a21898bb06" is_pre_election: true
I20240426 13:13:22.276402  4231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393e2bc65b24049a6542894a3a6a1d9" candidate_uuid: "0d9f076426a441f8b396f442cdb98f5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1149982f4cff4aa4ad64f110d25d5eed" is_pre_election: true
I20240426 13:13:22.276569  4080 raft_consensus.cc:2454] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9f076426a441f8b396f442cdb98f5b in term 0.
I20240426 13:13:22.276593  4231 raft_consensus.cc:2454] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d9f076426a441f8b396f442cdb98f5b in term 0.
I20240426 13:13:22.276896  4120 leader_election.cc:302] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9f076426a441f8b396f442cdb98f5b, 49e48ccc5c534611a13c30a21898bb06; no voters: 
I20240426 13:13:22.277235  4278 raft_consensus.cc:2781] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:13:22.277315  4278 raft_consensus.cc:482] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:13:22.277351  4278 raft_consensus.cc:3037] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:22.278203  4278 raft_consensus.cc:504] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:22.278523  4278 leader_election.cc:288] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [CANDIDATE]: Term 1 election: Requested vote from peers 49e48ccc5c534611a13c30a21898bb06 (127.0.99.129:40519), 1149982f4cff4aa4ad64f110d25d5eed (127.0.99.131:41967)
I20240426 13:13:22.278810  4231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393e2bc65b24049a6542894a3a6a1d9" candidate_uuid: "0d9f076426a441f8b396f442cdb98f5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1149982f4cff4aa4ad64f110d25d5eed"
I20240426 13:13:22.278889  4231 raft_consensus.cc:3037] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:22.278836  4080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393e2bc65b24049a6542894a3a6a1d9" candidate_uuid: "0d9f076426a441f8b396f442cdb98f5b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49e48ccc5c534611a13c30a21898bb06"
I20240426 13:13:22.278959  4080 raft_consensus.cc:3037] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:22.279619  4231 raft_consensus.cc:2454] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9f076426a441f8b396f442cdb98f5b in term 1.
I20240426 13:13:22.279760  4080 raft_consensus.cc:2454] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d9f076426a441f8b396f442cdb98f5b in term 1.
I20240426 13:13:22.279876  4119 leader_election.cc:302] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d9f076426a441f8b396f442cdb98f5b, 1149982f4cff4aa4ad64f110d25d5eed; no voters: 
I20240426 13:13:22.280045  4278 raft_consensus.cc:2781] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:22.280220  4278 raft_consensus.cc:686] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 1 LEADER]: Becoming Leader. State: Replica: 0d9f076426a441f8b396f442cdb98f5b, State: Running, Role: LEADER
I20240426 13:13:22.280373  4278 consensus_queue.cc:237] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:22.281777  3976 catalog_manager.cc:5484] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d9f076426a441f8b396f442cdb98f5b (127.0.99.130). New cstate: current_term: 1 leader_uuid: "0d9f076426a441f8b396f442cdb98f5b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } health_report { overall_health: UNKNOWN } } }
I20240426 13:13:22.326526  4231 raft_consensus.cc:1261] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 1 FOLLOWER]: Refusing update from remote peer 0d9f076426a441f8b396f442cdb98f5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:22.326601  4080 raft_consensus.cc:1261] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 1 FOLLOWER]: Refusing update from remote peer 0d9f076426a441f8b396f442cdb98f5b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:22.327083  4282 consensus_queue.cc:1035] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:22.327304  4278 consensus_queue.cc:1035] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:32.885540  4324 raft_consensus.cc:979] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b: : Instructing follower 49e48ccc5c534611a13c30a21898bb06 to start an election
I20240426 13:13:32.885632  4324 raft_consensus.cc:1067] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 1 LEADER]: Signalling peer 49e48ccc5c534611a13c30a21898bb06 to start an election
I20240426 13:13:32.886036  4080 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d393e2bc65b24049a6542894a3a6a1d9"
dest_uuid: "49e48ccc5c534611a13c30a21898bb06"
 from {username='slave'} at 127.0.0.1:38590
I20240426 13:13:32.886161  4080 raft_consensus.cc:482] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:13:32.886204  4080 raft_consensus.cc:3037] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:32.887799  4080 raft_consensus.cc:504] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:32.888202  4080 leader_election.cc:288] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [CANDIDATE]: Term 2 election: Requested vote from peers 0d9f076426a441f8b396f442cdb98f5b (127.0.99.130:37089), 1149982f4cff4aa4ad64f110d25d5eed (127.0.99.131:41967)
I20240426 13:13:32.891541  4231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393e2bc65b24049a6542894a3a6a1d9" candidate_uuid: "49e48ccc5c534611a13c30a21898bb06" candidate_term: 2 candidate_status { last_received { term: 1 index: 3065 } } ignore_live_leader: true dest_uuid: "1149982f4cff4aa4ad64f110d25d5eed"
I20240426 13:13:32.891647  4231 raft_consensus.cc:3037] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:32.891695  4156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d393e2bc65b24049a6542894a3a6a1d9" candidate_uuid: "49e48ccc5c534611a13c30a21898bb06" candidate_term: 2 candidate_status { last_received { term: 1 index: 3065 } } ignore_live_leader: true dest_uuid: "0d9f076426a441f8b396f442cdb98f5b"
I20240426 13:13:32.891777  4156 raft_consensus.cc:3032] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:13:32.891825  4156 raft_consensus.cc:729] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0d9f076426a441f8b396f442cdb98f5b, State: Running, Role: LEADER
I20240426 13:13:32.891959  4156 consensus_queue.cc:260] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3065, Committed index: 3065, Last appended: 1.3065, Last appended by leader: 3065, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:32.892141  4156 raft_consensus.cc:3037] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:13:32.892691  4231 raft_consensus.cc:2454] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e48ccc5c534611a13c30a21898bb06 in term 2.
I20240426 13:13:32.893056  4043 leader_election.cc:302] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1149982f4cff4aa4ad64f110d25d5eed, 49e48ccc5c534611a13c30a21898bb06; no voters: 
I20240426 13:13:32.893097  4156 raft_consensus.cc:2454] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 49e48ccc5c534611a13c30a21898bb06 in term 2.
I20240426 13:13:32.893453  4333 raft_consensus.cc:2781] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:13:32.893625  4333 raft_consensus.cc:686] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 LEADER]: Becoming Leader. State: Replica: 49e48ccc5c534611a13c30a21898bb06, State: Running, Role: LEADER
I20240426 13:13:32.893785  4333 consensus_queue.cc:237] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3065, Committed index: 3065, Last appended: 1.3065, Last appended by leader: 3065, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:32.895102  3975 catalog_manager.cc:5484] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 reported cstate change: term changed from 1 to 2, leader changed from 0d9f076426a441f8b396f442cdb98f5b (127.0.99.130) to 49e48ccc5c534611a13c30a21898bb06 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "49e48ccc5c534611a13c30a21898bb06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } health_report { overall_health: UNKNOWN } } }
I20240426 13:13:32.903026   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:13:32.906438   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:32.906581   398 tablet_replica.cc:330] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: stopping tablet replica
I20240426 13:13:32.906683   398 raft_consensus.cc:2229] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:13:32.906801   398 pending_rounds.cc:62] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Trying to abort 1 pending ops.
I20240426 13:13:32.906827   398 pending_rounds.cc:68] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Aborting op as it isn't in flight: id { term: 2 index: 3066 } timestamp: 7021106024014503936 op_type: NO_OP noop_request { }
I20240426 13:13:32.906868   398 raft_consensus.cc:2866] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240426 13:13:32.906906   398 raft_consensus.cc:2258] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:32.922997   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:13:32.924669   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:32.927245  4339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:32.928180  4342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:32.929000  4340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:32.929824   398 server_base.cc:1034] running on GCE node
I20240426 13:13:32.930022   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:32.930068   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:32.930089   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137212930089 us; error 0 us; skew 500 ppm
I20240426 13:13:32.930986   398 webserver.cc:460] Webserver started at http://127.0.99.129:44999/ using document root <none> and password file <none>
I20240426 13:13:32.931119   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:32.931157   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:13:32.931988   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:32.932754  4347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:32.932973   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:32.933037   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "49e48ccc5c534611a13c30a21898bb06"
format_stamp: "Formatted at 2024-04-26 13:13:21 on dist-test-slave-2jsl"
I20240426 13:13:32.933110   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletBootstrapping.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:32.941463   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:32.941691   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:32.941898   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:32.942524  4354 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240426 13:13:32.943738   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:13:32.943781   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:13:32.943825   398 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240426 13:13:32.944391   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:13:32.944423   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
W20240426 13:13:32.944420  4354 ts_tablet_manager.cc:1346] Injecting 1000ms delay in tablet bootstrapping
I20240426 13:13:32.947958   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:40519
I20240426 13:13:32.948374  4411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:40519 every 8 connection(s)
I20240426 13:13:32.953702  4412 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35873
I20240426 13:13:32.953789  4412 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:32.953985  4412 heartbeater.cc:511] Master 127.0.99.190:35873 requested a full tablet report, sending...
I20240426 13:13:32.954854   398 tablet_server.cc:178] TabletServer@127.0.99.129:40519 shutting down...
I20240426 13:13:32.955027  3975 ts_manager.cc:194] Re-registered known tserver with Master: 49e48ccc5c534611a13c30a21898bb06 (127.0.99.129:40519)
I20240426 13:13:32.956027  3975 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36624
I20240426 13:13:32.957773   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:33.944793  4354 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Bootstrap starting.
W20240426 13:13:34.700721  4419 raft_consensus.cc:659] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b: failed to trigger leader election: Illegal state: leader elections are disabled
I20240426 13:13:38.129468  4354 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Bootstrap replayed 1/1 log segments. Stats: ops{read=3066 overwritten=0 applied=3065 ignored=0} inserts{seen=153200 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20240426 13:13:38.129937  4354 tablet_bootstrap.cc:492] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Bootstrap complete.
I20240426 13:13:38.131153  4354 ts_tablet_manager.cc:1367] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Time spent bootstrapping tablet: real 4.186s	user 4.100s	sys 0.084s
I20240426 13:13:38.131528  4354 raft_consensus.cc:348] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:38.131682  4354 raft_consensus.cc:729] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e48ccc5c534611a13c30a21898bb06, State: Initialized, Role: FOLLOWER
I20240426 13:13:38.131801  4354 consensus_queue.cc:260] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3065, Last appended: 2.3066, Last appended by leader: 3066, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49e48ccc5c534611a13c30a21898bb06" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 40519 } } peers { permanent_uuid: "0d9f076426a441f8b396f442cdb98f5b" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 37089 } } peers { permanent_uuid: "1149982f4cff4aa4ad64f110d25d5eed" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41967 } }
I20240426 13:13:38.132051  4354 ts_tablet_manager.cc:1398] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:13:38.132303   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:13:38.132411   398 raft_consensus.cc:2229] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:38.132477   398 pending_rounds.cc:62] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Trying to abort 1 pending ops.
I20240426 13:13:38.132505   398 pending_rounds.cc:68] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06: Aborting op as it isn't in flight: id { term: 2 index: 3066 } timestamp: 7021106024014503936 op_type: NO_OP noop_request { }
I20240426 13:13:38.132558   398 raft_consensus.cc:2866] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20240426 13:13:38.132632   398 raft_consensus.cc:2258] T d393e2bc65b24049a6542894a3a6a1d9 P 49e48ccc5c534611a13c30a21898bb06 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:38.147516   398 tablet_server.cc:194] TabletServer@127.0.99.129:40519 shutdown complete.
I20240426 13:13:38.149098   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:13:38.152357   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:38.152483   398 tablet_replica.cc:330] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b: stopping tablet replica
I20240426 13:13:38.152570   398 raft_consensus.cc:2229] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:38.152657   398 raft_consensus.cc:2258] T d393e2bc65b24049a6542894a3a6a1d9 P 0d9f076426a441f8b396f442cdb98f5b [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:38.167768   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:13:38.169421   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:13:38.173409   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:38.173543   398 tablet_replica.cc:330] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed: stopping tablet replica
I20240426 13:13:38.173636   398 raft_consensus.cc:2229] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:38.173758   398 raft_consensus.cc:2258] T d393e2bc65b24049a6542894a3a6a1d9 P 1149982f4cff4aa4ad64f110d25d5eed [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:38.188751   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:13:38.190515   398 master.cc:537] Master@127.0.99.190:35873 shutting down...
I20240426 13:13:38.194123   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:13:38.194234   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:38.194289   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P b506b67485bd48659f1fbc5db7e11a1a: stopping tablet replica
I20240426 13:13:38.205917   398 master.cc:552] Master@127.0.99.190:35873 shutdown complete.
[       OK ] TxnParticipantITest.TestProxyTabletBootstrapping (16430 ms)
[ RUN      ] TxnParticipantITest.TestProxyTabletNotRunning
I20240426 13:13:38.213937   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:37975
I20240426 13:13:38.214226   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240426 13:13:38.216629   398 server_base.cc:1034] running on GCE node
W20240426 13:13:38.217551  4427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:38.218288  4426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:38.219060  4429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:38.219225   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:38.219262   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:38.219275   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137218219275 us; error 0 us; skew 500 ppm
I20240426 13:13:38.225927   398 webserver.cc:460] Webserver started at http://127.0.99.190:37841/ using document root <none> and password file <none>
I20240426 13:13:38.226087   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:38.226127   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:38.226192   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:38.226507   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c"
format_stamp: "Formatted at 2024-04-26 13:13:38 on dist-test-slave-2jsl"
I20240426 13:13:38.227671   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:13:38.228377  4434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.228580   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:13:38.228628   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c"
format_stamp: "Formatted at 2024-04-26 13:13:38 on dist-test-slave-2jsl"
I20240426 13:13:38.228672   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:38.235209   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:38.235423   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:38.238682   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:37975
I20240426 13:13:38.243292  4486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:37975 every 8 connection(s)
I20240426 13:13:38.243605  4487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:38.245028  4487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c: Bootstrap starting.
I20240426 13:13:38.245595  4487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:38.246357  4487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c: No bootstrap required, opened a new log
I20240426 13:13:38.246639  4487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c" member_type: VOTER }
I20240426 13:13:38.246708  4487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:38.246742  4487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bedc0a0c2be4d51b8cb158cc6ef856c, State: Initialized, Role: FOLLOWER
I20240426 13:13:38.246837  4487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c" member_type: VOTER }
I20240426 13:13:38.246892  4487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:13:38.246923  4487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:13:38.246966  4487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:38.247545  4487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c" member_type: VOTER }
I20240426 13:13:38.247661  4487 leader_election.cc:302] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bedc0a0c2be4d51b8cb158cc6ef856c; no voters: 
I20240426 13:13:38.247814  4487 leader_election.cc:288] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:13:38.247896  4490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:38.248080  4490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 1 LEADER]: Becoming Leader. State: Replica: 6bedc0a0c2be4d51b8cb158cc6ef856c, State: Running, Role: LEADER
I20240426 13:13:38.248160  4487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:13:38.248205  4490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c" member_type: VOTER }
I20240426 13:13:38.248587  4492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bedc0a0c2be4d51b8cb158cc6ef856c. Latest consensus state: current_term: 1 leader_uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c" member_type: VOTER } }
I20240426 13:13:38.248564  4491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bedc0a0c2be4d51b8cb158cc6ef856c" member_type: VOTER } }
I20240426 13:13:38.248653  4492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:38.248663  4491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:38.248891  4495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:13:38.249491  4495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:13:38.249725   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:13:38.250926  4495 catalog_manager.cc:1347] Generated new cluster ID: a091253998464401b1f1fccc9d68abc0
I20240426 13:13:38.250978  4495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:13:38.263079  4495 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:13:38.263535  4495 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:13:38.270457  4495 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c: Generated new TSK 0
I20240426 13:13:38.270577  4495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:13:38.281679   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:38.283965  4508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:38.284848  4509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:38.285641  4511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:38.286432   398 server_base.cc:1034] running on GCE node
I20240426 13:13:38.286545   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:38.286572   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:38.286584   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137218286585 us; error 0 us; skew 500 ppm
I20240426 13:13:38.287276   398 webserver.cc:460] Webserver started at http://127.0.99.129:35555/ using document root <none> and password file <none>
I20240426 13:13:38.287374   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:38.287415   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:38.287460   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:38.287724   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "92ae577f50124f3b9447bad189f2feca"
format_stamp: "Formatted at 2024-04-26 13:13:38 on dist-test-slave-2jsl"
I20240426 13:13:38.288847   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:38.289543  4516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.289741   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:13:38.289793   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "92ae577f50124f3b9447bad189f2feca"
format_stamp: "Formatted at 2024-04-26 13:13:38 on dist-test-slave-2jsl"
I20240426 13:13:38.289845   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:38.300948   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:38.301178   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:38.301381   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:38.301719   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:38.301753   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.301793   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:38.301818   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.305179   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:38785
I20240426 13:13:38.305189  4579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:38785 every 8 connection(s)
I20240426 13:13:38.305863   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:38.307790  4585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:38.308444  4584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:38.309111  4587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:38.309217  4580 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37975
I20240426 13:13:38.309288  4580 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:38.309466  4580 heartbeater.cc:511] Master 127.0.99.190:37975 requested a full tablet report, sending...
I20240426 13:13:38.309790   398 server_base.cc:1034] running on GCE node
I20240426 13:13:38.309932   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:38.309984   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:38.310011   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137218310011 us; error 0 us; skew 500 ppm
I20240426 13:13:38.310050  4451 ts_manager.cc:194] Registered new tserver with Master: 92ae577f50124f3b9447bad189f2feca (127.0.99.129:38785)
I20240426 13:13:38.310783   398 webserver.cc:460] Webserver started at http://127.0.99.130:45563/ using document root <none> and password file <none>
I20240426 13:13:38.310887   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:38.310933   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:38.310976   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:38.311111  4451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36178
I20240426 13:13:38.311261   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "591e4b7a66274204af14c1a696ac5054"
format_stamp: "Formatted at 2024-04-26 13:13:38 on dist-test-slave-2jsl"
I20240426 13:13:38.312547   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:13:38.313277  4592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.313459   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:13:38.313517   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "591e4b7a66274204af14c1a696ac5054"
format_stamp: "Formatted at 2024-04-26 13:13:38 on dist-test-slave-2jsl"
I20240426 13:13:38.313576   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:38.329769   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:38.330053   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:38.330251   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:38.330629   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:38.330662   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.330701   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:38.330731   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.334556   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:39501
I20240426 13:13:38.334580  4655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:39501 every 8 connection(s)
I20240426 13:13:38.335147   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:38.337046  4659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:38.337792  4660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:38.338081  4656 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37975
I20240426 13:13:38.338142  4656 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:38.338266  4656 heartbeater.cc:511] Master 127.0.99.190:37975 requested a full tablet report, sending...
W20240426 13:13:38.338537  4662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:38.338711  4451 ts_manager.cc:194] Registered new tserver with Master: 591e4b7a66274204af14c1a696ac5054 (127.0.99.130:39501)
I20240426 13:13:38.339104  4451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36194
I20240426 13:13:38.339231   398 server_base.cc:1034] running on GCE node
I20240426 13:13:38.339375   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:38.339418   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:38.339438   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137218339437 us; error 0 us; skew 500 ppm
I20240426 13:13:38.340085   398 webserver.cc:460] Webserver started at http://127.0.99.131:45541/ using document root <none> and password file <none>
I20240426 13:13:38.340190   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:38.340234   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:38.340298   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:38.340559   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "afa173bb762446289d04928d04e28d44"
format_stamp: "Formatted at 2024-04-26 13:13:38 on dist-test-slave-2jsl"
I20240426 13:13:38.341619   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:38.342342  4667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.342519   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.342576   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "afa173bb762446289d04928d04e28d44"
format_stamp: "Formatted at 2024-04-26 13:13:38 on dist-test-slave-2jsl"
I20240426 13:13:38.342649   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:38.352952   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:38.353204   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:38.353415   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:38.353771   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:38.353807   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.353861   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:38.353897   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:38.357687   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:35849
I20240426 13:13:38.357728  4730 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:35849 every 8 connection(s)
I20240426 13:13:38.361140  4731 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37975
I20240426 13:13:38.361202  4731 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:38.361336  4731 heartbeater.cc:511] Master 127.0.99.190:37975 requested a full tablet report, sending...
I20240426 13:13:38.361776  4451 ts_manager.cc:194] Registered new tserver with Master: afa173bb762446289d04928d04e28d44 (127.0.99.131:35849)
I20240426 13:13:38.362217  4451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36202
I20240426 13:13:38.362452   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00443317s
I20240426 13:13:38.362898   398 test_util.cc:268] Using random seed: -1409339887
I20240426 13:13:38.368600  4451 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36216:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:13:38.368886  4451 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:13:38.376443  4544 tablet_service.cc:1467] Processing CreateTablet for tablet 112ff66f2ede4d749077bfd4ed2bf17b (DEFAULT_TABLE table=test-workload [id=947b54659b144ceab666482d41c554e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:38.376698  4544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112ff66f2ede4d749077bfd4ed2bf17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:38.376914  4620 tablet_service.cc:1467] Processing CreateTablet for tablet 112ff66f2ede4d749077bfd4ed2bf17b (DEFAULT_TABLE table=test-workload [id=947b54659b144ceab666482d41c554e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:38.377110  4620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112ff66f2ede4d749077bfd4ed2bf17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:38.378417  4695 tablet_service.cc:1467] Processing CreateTablet for tablet 112ff66f2ede4d749077bfd4ed2bf17b (DEFAULT_TABLE table=test-workload [id=947b54659b144ceab666482d41c554e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:38.378647  4695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 112ff66f2ede4d749077bfd4ed2bf17b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:38.379416  4747 tablet_bootstrap.cc:492] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: Bootstrap starting.
I20240426 13:13:38.380246  4747 tablet_bootstrap.cc:654] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:38.380543  4748 tablet_bootstrap.cc:492] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44: Bootstrap starting.
I20240426 13:13:38.380924  4746 tablet_bootstrap.cc:492] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054: Bootstrap starting.
I20240426 13:13:38.381208  4747 tablet_bootstrap.cc:492] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: No bootstrap required, opened a new log
I20240426 13:13:38.381283  4748 tablet_bootstrap.cc:654] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:38.381294  4747 ts_tablet_manager.cc:1367] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:13:38.381701  4746 tablet_bootstrap.cc:654] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:38.381664  4747 raft_consensus.cc:348] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.381805  4747 raft_consensus.cc:374] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:38.381839  4747 raft_consensus.cc:729] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92ae577f50124f3b9447bad189f2feca, State: Initialized, Role: FOLLOWER
I20240426 13:13:38.382019  4747 consensus_queue.cc:260] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.382122  4748 tablet_bootstrap.cc:492] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44: No bootstrap required, opened a new log
I20240426 13:13:38.382218  4748 ts_tablet_manager.cc:1367] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:13:38.382380  4747 ts_tablet_manager.cc:1398] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:38.382511  4580 heartbeater.cc:503] Master 127.0.99.190:37975 was elected leader, sending a full tablet report...
I20240426 13:13:38.382725  4748 raft_consensus.cc:348] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.382812  4748 raft_consensus.cc:374] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:38.382841  4748 raft_consensus.cc:729] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa173bb762446289d04928d04e28d44, State: Initialized, Role: FOLLOWER
I20240426 13:13:38.383080  4746 tablet_bootstrap.cc:492] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054: No bootstrap required, opened a new log
I20240426 13:13:38.383081  4748 consensus_queue.cc:260] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.383200  4746 ts_tablet_manager.cc:1367] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:13:38.383431  4731 heartbeater.cc:503] Master 127.0.99.190:37975 was elected leader, sending a full tablet report...
I20240426 13:13:38.383442  4748 ts_tablet_manager.cc:1398] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:38.383615  4746 raft_consensus.cc:348] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.383759  4746 raft_consensus.cc:374] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:38.383806  4746 raft_consensus.cc:729] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e4b7a66274204af14c1a696ac5054, State: Initialized, Role: FOLLOWER
I20240426 13:13:38.383934  4746 consensus_queue.cc:260] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.384231  4746 ts_tablet_manager.cc:1398] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:38.384564  4656 heartbeater.cc:503] Master 127.0.99.190:37975 was elected leader, sending a full tablet report...
I20240426 13:13:38.634902  4751 raft_consensus.cc:482] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:13:38.635072  4751 raft_consensus.cc:504] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.635654  4751 leader_election.cc:288] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 591e4b7a66274204af14c1a696ac5054 (127.0.99.130:39501), afa173bb762446289d04928d04e28d44 (127.0.99.131:35849)
I20240426 13:13:38.639366  4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112ff66f2ede4d749077bfd4ed2bf17b" candidate_uuid: "92ae577f50124f3b9447bad189f2feca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e4b7a66274204af14c1a696ac5054" is_pre_election: true
I20240426 13:13:38.639405  4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112ff66f2ede4d749077bfd4ed2bf17b" candidate_uuid: "92ae577f50124f3b9447bad189f2feca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa173bb762446289d04928d04e28d44" is_pre_election: true
I20240426 13:13:38.639482  4625 raft_consensus.cc:2454] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ae577f50124f3b9447bad189f2feca in term 0.
I20240426 13:13:38.639498  4705 raft_consensus.cc:2454] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92ae577f50124f3b9447bad189f2feca in term 0.
I20240426 13:13:38.639737  4519 leader_election.cc:302] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 591e4b7a66274204af14c1a696ac5054, 92ae577f50124f3b9447bad189f2feca; no voters: 
I20240426 13:13:38.639920  4751 raft_consensus.cc:2781] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:13:38.639971  4751 raft_consensus.cc:482] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:13:38.639997  4751 raft_consensus.cc:3037] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:38.640863  4751 raft_consensus.cc:504] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.641170  4751 leader_election.cc:288] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [CANDIDATE]: Term 1 election: Requested vote from peers 591e4b7a66274204af14c1a696ac5054 (127.0.99.130:39501), afa173bb762446289d04928d04e28d44 (127.0.99.131:35849)
I20240426 13:13:38.641427  4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112ff66f2ede4d749077bfd4ed2bf17b" candidate_uuid: "92ae577f50124f3b9447bad189f2feca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "591e4b7a66274204af14c1a696ac5054"
I20240426 13:13:38.641518  4625 raft_consensus.cc:3037] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:38.641500  4705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "112ff66f2ede4d749077bfd4ed2bf17b" candidate_uuid: "92ae577f50124f3b9447bad189f2feca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afa173bb762446289d04928d04e28d44"
I20240426 13:13:38.641659  4705 raft_consensus.cc:3037] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:38.642424  4625 raft_consensus.cc:2454] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ae577f50124f3b9447bad189f2feca in term 1.
I20240426 13:13:38.642516  4705 raft_consensus.cc:2454] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92ae577f50124f3b9447bad189f2feca in term 1.
I20240426 13:13:38.642664  4519 leader_election.cc:302] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 591e4b7a66274204af14c1a696ac5054, 92ae577f50124f3b9447bad189f2feca; no voters: 
I20240426 13:13:38.642827  4751 raft_consensus.cc:2781] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:38.643007  4751 raft_consensus.cc:686] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 1 LEADER]: Becoming Leader. State: Replica: 92ae577f50124f3b9447bad189f2feca, State: Running, Role: LEADER
I20240426 13:13:38.643148  4751 consensus_queue.cc:237] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } }
I20240426 13:13:38.644567  4451 catalog_manager.cc:5484] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca reported cstate change: term changed from 0 to 1, leader changed from <none> to 92ae577f50124f3b9447bad189f2feca (127.0.99.129). New cstate: current_term: 1 leader_uuid: "92ae577f50124f3b9447bad189f2feca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92ae577f50124f3b9447bad189f2feca" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 } health_report { overall_health: UNKNOWN } } }
I20240426 13:13:38.675864  4705 raft_consensus.cc:1261] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 1 FOLLOWER]: Refusing update from remote peer 92ae577f50124f3b9447bad189f2feca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:38.675864  4625 raft_consensus.cc:1261] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 1 FOLLOWER]: Refusing update from remote peer 92ae577f50124f3b9447bad189f2feca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:38.676241  4751 consensus_queue.cc:1035] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [LEADER]: Connected to new peer: Peer: permanent_uuid: "591e4b7a66274204af14c1a696ac5054" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:38.676461  4756 consensus_queue.cc:1035] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [LEADER]: Connected to new peer: Peer: permanent_uuid: "afa173bb762446289d04928d04e28d44" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:48.677428   398 tablet_replica.cc:330] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: stopping tablet replica
I20240426 13:13:48.677536   398 raft_consensus.cc:2229] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:13:48.677665   398 raft_consensus.cc:2258] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:48.680889   398 ts_tablet_manager.cc:1875] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 13:13:48.682873   398 ts_tablet_manager.cc:1888] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2977
I20240426 13:13:48.682946   398 log.cc:1196] T 112ff66f2ede4d749077bfd4ed2bf17b P 92ae577f50124f3b9447bad189f2feca: Deleting WAL directory at /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotRunning.1714137102075184-398-0/minicluster-data/ts-0-root/wals/112ff66f2ede4d749077bfd4ed2bf17b
I20240426 13:13:48.688122   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:13:48.692291   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:48.705744   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:13:48.707414   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:13:48.710953   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:48.711074   398 tablet_replica.cc:330] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054: stopping tablet replica
I20240426 13:13:48.711167   398 raft_consensus.cc:2229] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:48.711266   398 raft_consensus.cc:2258] T 112ff66f2ede4d749077bfd4ed2bf17b P 591e4b7a66274204af14c1a696ac5054 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:48.727631   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:13:48.729125   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:13:48.733026   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:48.733192   398 tablet_replica.cc:330] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44: stopping tablet replica
I20240426 13:13:48.733281   398 raft_consensus.cc:2229] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:48.733392   398 raft_consensus.cc:2258] T 112ff66f2ede4d749077bfd4ed2bf17b P afa173bb762446289d04928d04e28d44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:48.738624   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:13:48.739954   398 master.cc:537] Master@127.0.99.190:37975 shutting down...
I20240426 13:13:48.743072   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:13:48.743163   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:48.743206   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6bedc0a0c2be4d51b8cb158cc6ef856c: stopping tablet replica
I20240426 13:13:48.754974   398 master.cc:552] Master@127.0.99.190:37975 shutdown complete.
[       OK ] TxnParticipantITest.TestProxyTabletNotRunning (10548 ms)
[ RUN      ] TxnParticipantITest.TestProxyTabletNotFound
I20240426 13:13:48.762020   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:33059
I20240426 13:13:48.762279   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:48.764765  4802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:48.765702  4803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:48.766556   398 server_base.cc:1034] running on GCE node
W20240426 13:13:48.767292  4805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:48.767467   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:48.767509   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:48.767524   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137228767524 us; error 0 us; skew 500 ppm
I20240426 13:13:48.768287   398 webserver.cc:460] Webserver started at http://127.0.99.190:38275/ using document root <none> and password file <none>
I20240426 13:13:48.768424   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:48.768486   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:48.768555   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:48.768880   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "f89abcb265754c5b9f4472bb443f80d3"
format_stamp: "Formatted at 2024-04-26 13:13:48 on dist-test-slave-2jsl"
I20240426 13:13:48.770143   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.001s
I20240426 13:13:48.770884  4810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.771075   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:13:48.771138   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "f89abcb265754c5b9f4472bb443f80d3"
format_stamp: "Formatted at 2024-04-26 13:13:48 on dist-test-slave-2jsl"
I20240426 13:13:48.771200   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:48.779528   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:48.779789   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:48.783063   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:33059
I20240426 13:13:48.789345  4863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:48.789363  4862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:33059 every 8 connection(s)
I20240426 13:13:48.790903  4863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3: Bootstrap starting.
I20240426 13:13:48.791545  4863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:48.792413  4863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3: No bootstrap required, opened a new log
I20240426 13:13:48.792726  4863 raft_consensus.cc:348] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89abcb265754c5b9f4472bb443f80d3" member_type: VOTER }
I20240426 13:13:48.792798  4863 raft_consensus.cc:374] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:48.792829  4863 raft_consensus.cc:729] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89abcb265754c5b9f4472bb443f80d3, State: Initialized, Role: FOLLOWER
I20240426 13:13:48.792944  4863 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89abcb265754c5b9f4472bb443f80d3" member_type: VOTER }
I20240426 13:13:48.793000  4863 raft_consensus.cc:388] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:13:48.793022  4863 raft_consensus.cc:482] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:13:48.793066  4863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:48.793625  4863 raft_consensus.cc:504] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89abcb265754c5b9f4472bb443f80d3" member_type: VOTER }
I20240426 13:13:48.793752  4863 leader_election.cc:302] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f89abcb265754c5b9f4472bb443f80d3; no voters: 
I20240426 13:13:48.793913  4863 leader_election.cc:288] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:13:48.794030  4866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:48.794230  4866 raft_consensus.cc:686] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 1 LEADER]: Becoming Leader. State: Replica: f89abcb265754c5b9f4472bb443f80d3, State: Running, Role: LEADER
I20240426 13:13:48.794315  4863 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:13:48.794355  4866 consensus_queue.cc:237] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89abcb265754c5b9f4472bb443f80d3" member_type: VOTER }
I20240426 13:13:48.794747  4867 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f89abcb265754c5b9f4472bb443f80d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89abcb265754c5b9f4472bb443f80d3" member_type: VOTER } }
I20240426 13:13:48.794802  4868 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89abcb265754c5b9f4472bb443f80d3. Latest consensus state: current_term: 1 leader_uuid: "f89abcb265754c5b9f4472bb443f80d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89abcb265754c5b9f4472bb443f80d3" member_type: VOTER } }
I20240426 13:13:48.794950  4868 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:48.794935  4867 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:48.795565  4874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:13:48.796197  4874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:13:48.796453   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:13:48.797644  4874 catalog_manager.cc:1347] Generated new cluster ID: cc3740a831c744b7b2a83dd08dcbf060
I20240426 13:13:48.797696  4874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:13:48.802153  4874 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:13:48.802582  4874 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:13:48.806344  4874 catalog_manager.cc:5857] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3: Generated new TSK 0
I20240426 13:13:48.806478  4874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:13:48.812229   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:48.814535  4885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:48.815316  4884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:48.816100  4887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:48.816922   398 server_base.cc:1034] running on GCE node
I20240426 13:13:48.817059   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:48.817091   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:48.817103   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137228817104 us; error 0 us; skew 500 ppm
I20240426 13:13:48.817776   398 webserver.cc:460] Webserver started at http://127.0.99.129:35573/ using document root <none> and password file <none>
I20240426 13:13:48.817886   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:48.817924   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:48.818007   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:48.818285   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "b46991dd328544edb5af2c36e9b1045b"
format_stamp: "Formatted at 2024-04-26 13:13:48 on dist-test-slave-2jsl"
I20240426 13:13:48.819437   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:48.820197  4892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.820411   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:48.820472   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "b46991dd328544edb5af2c36e9b1045b"
format_stamp: "Formatted at 2024-04-26 13:13:48 on dist-test-slave-2jsl"
I20240426 13:13:48.820541   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:48.826252   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:48.826505   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:48.826721   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:48.827078   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:48.827111   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.827152   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:48.827188   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.830518   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:45357
I20240426 13:13:48.830550  4955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:45357 every 8 connection(s)
I20240426 13:13:48.831099   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:48.833115  4960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:48.833952  4961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:48.834834  4963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:48.835537   398 server_base.cc:1034] running on GCE node
I20240426 13:13:48.835552  4956 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33059
I20240426 13:13:48.835655  4956 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:48.835713   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:48.835749   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:48.835772   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137228835772 us; error 0 us; skew 500 ppm
I20240426 13:13:48.835841  4956 heartbeater.cc:511] Master 127.0.99.190:33059 requested a full tablet report, sending...
I20240426 13:13:48.836410  4827 ts_manager.cc:194] Registered new tserver with Master: b46991dd328544edb5af2c36e9b1045b (127.0.99.129:45357)
I20240426 13:13:48.836580   398 webserver.cc:460] Webserver started at http://127.0.99.130:40921/ using document root <none> and password file <none>
I20240426 13:13:48.836712   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:48.836750   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:48.836803   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:48.837090   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "d646fe009ae84b7d9e47e438176315d9"
format_stamp: "Formatted at 2024-04-26 13:13:48 on dist-test-slave-2jsl"
I20240426 13:13:48.837175  4827 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39204
I20240426 13:13:48.838445   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:13:48.839190  4968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.839417   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:48.839473   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "d646fe009ae84b7d9e47e438176315d9"
format_stamp: "Formatted at 2024-04-26 13:13:48 on dist-test-slave-2jsl"
I20240426 13:13:48.839530   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:48.848345   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:48.848593   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:48.848815   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:48.849177   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:48.849211   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.849251   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:48.849283   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.852614   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:33045
I20240426 13:13:48.852650  5031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:33045 every 8 connection(s)
I20240426 13:13:48.853232   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:48.855444  5035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:48.856178  5036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:48.856464  5032 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33059
I20240426 13:13:48.856544  5032 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:48.856686  5032 heartbeater.cc:511] Master 127.0.99.190:33059 requested a full tablet report, sending...
W20240426 13:13:48.856940  5038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:48.857292  4827 ts_manager.cc:194] Registered new tserver with Master: d646fe009ae84b7d9e47e438176315d9 (127.0.99.130:33045)
I20240426 13:13:48.857669   398 server_base.cc:1034] running on GCE node
I20240426 13:13:48.857793  4827 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39208
I20240426 13:13:48.857797   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:48.857880   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:48.857903   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137228857903 us; error 0 us; skew 500 ppm
I20240426 13:13:48.858664   398 webserver.cc:460] Webserver started at http://127.0.99.131:41655/ using document root <none> and password file <none>
I20240426 13:13:48.858757   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:48.858789   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:48.858834   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:48.859107   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "f0f724fb25524a798851a7890b50e9cf"
format_stamp: "Formatted at 2024-04-26 13:13:48 on dist-test-slave-2jsl"
I20240426 13:13:48.860215   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:48.860914  5043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.861101   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:13:48.861152   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "f0f724fb25524a798851a7890b50e9cf"
format_stamp: "Formatted at 2024-04-26 13:13:48 on dist-test-slave-2jsl"
I20240426 13:13:48.861197   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestProxyTabletNotFound.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:48.873562   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:48.873797   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:48.874011   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:48.874362   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:48.874397   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.874430   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:48.874450   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:48.878271   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:41477
I20240426 13:13:48.878327  5106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:41477 every 8 connection(s)
I20240426 13:13:48.881711  5107 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33059
I20240426 13:13:48.881794  5107 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:48.881959  5107 heartbeater.cc:511] Master 127.0.99.190:33059 requested a full tablet report, sending...
I20240426 13:13:48.882489  4827 ts_manager.cc:194] Registered new tserver with Master: f0f724fb25524a798851a7890b50e9cf (127.0.99.131:41477)
I20240426 13:13:48.882920  4827 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39224
I20240426 13:13:48.883137   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004521361s
I20240426 13:13:48.883515   398 test_util.cc:268] Using random seed: -1398819270
I20240426 13:13:48.889685  4827 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39226:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:13:48.890072  4827 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:13:48.898404  5071 tablet_service.cc:1467] Processing CreateTablet for tablet 026f8a8e25ee44ffb5078a9893a18c54 (DEFAULT_TABLE table=test-workload [id=88eb53146bf04602b3a026f27708def2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:48.898546  4996 tablet_service.cc:1467] Processing CreateTablet for tablet 026f8a8e25ee44ffb5078a9893a18c54 (DEFAULT_TABLE table=test-workload [id=88eb53146bf04602b3a026f27708def2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:48.898617  4920 tablet_service.cc:1467] Processing CreateTablet for tablet 026f8a8e25ee44ffb5078a9893a18c54 (DEFAULT_TABLE table=test-workload [id=88eb53146bf04602b3a026f27708def2]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:48.898667  5071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026f8a8e25ee44ffb5078a9893a18c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:48.898792  4920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026f8a8e25ee44ffb5078a9893a18c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:48.898792  4996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 026f8a8e25ee44ffb5078a9893a18c54. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:48.900876  5122 tablet_bootstrap.cc:492] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b: Bootstrap starting.
I20240426 13:13:48.901010  5123 tablet_bootstrap.cc:492] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9: Bootstrap starting.
I20240426 13:13:48.901726  5123 tablet_bootstrap.cc:654] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:48.901726  5122 tablet_bootstrap.cc:654] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:48.902098  5124 tablet_bootstrap.cc:492] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf: Bootstrap starting.
I20240426 13:13:48.902854  5123 tablet_bootstrap.cc:492] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9: No bootstrap required, opened a new log
I20240426 13:13:48.902956  5124 tablet_bootstrap.cc:654] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:48.902983  5123 ts_tablet_manager.cc:1367] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:13:48.903053  5122 tablet_bootstrap.cc:492] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b: No bootstrap required, opened a new log
I20240426 13:13:48.903136  5122 ts_tablet_manager.cc:1367] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:13:48.903388  5123 raft_consensus.cc:348] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:48.903483  5123 raft_consensus.cc:374] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:48.903479  5122 raft_consensus.cc:348] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:48.903574  5123 raft_consensus.cc:729] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d646fe009ae84b7d9e47e438176315d9, State: Initialized, Role: FOLLOWER
I20240426 13:13:48.903604  5122 raft_consensus.cc:374] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:48.903647  5122 raft_consensus.cc:729] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b46991dd328544edb5af2c36e9b1045b, State: Initialized, Role: FOLLOWER
I20240426 13:13:48.903757  5123 consensus_queue.cc:260] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:48.903797  5122 consensus_queue.cc:260] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:48.903926  5124 tablet_bootstrap.cc:492] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf: No bootstrap required, opened a new log
I20240426 13:13:48.903992  5124 ts_tablet_manager.cc:1367] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:13:48.904110  5032 heartbeater.cc:503] Master 127.0.99.190:33059 was elected leader, sending a full tablet report...
I20240426 13:13:48.904107  5122 ts_tablet_manager.cc:1398] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:48.904109  5123 ts_tablet_manager.cc:1398] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:48.904289  4956 heartbeater.cc:503] Master 127.0.99.190:33059 was elected leader, sending a full tablet report...
I20240426 13:13:48.904376  5124 raft_consensus.cc:348] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:48.904537  5124 raft_consensus.cc:374] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:48.904623  5124 raft_consensus.cc:729] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0f724fb25524a798851a7890b50e9cf, State: Initialized, Role: FOLLOWER
I20240426 13:13:48.904769  5124 consensus_queue.cc:260] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:48.905146  5124 ts_tablet_manager.cc:1398] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:48.905151  5107 heartbeater.cc:503] Master 127.0.99.190:33059 was elected leader, sending a full tablet report...
W20240426 13:13:49.081074  4957 tablet.cc:2369] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 13:13:49.086555  5129 raft_consensus.cc:482] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:13:49.086649  5129 raft_consensus.cc:504] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:49.087013  5129 leader_election.cc:288] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d646fe009ae84b7d9e47e438176315d9 (127.0.99.130:33045), f0f724fb25524a798851a7890b50e9cf (127.0.99.131:41477)
I20240426 13:13:49.090055  5006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026f8a8e25ee44ffb5078a9893a18c54" candidate_uuid: "b46991dd328544edb5af2c36e9b1045b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646fe009ae84b7d9e47e438176315d9" is_pre_election: true
I20240426 13:13:49.090188  5006 raft_consensus.cc:2454] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b46991dd328544edb5af2c36e9b1045b in term 0.
I20240426 13:13:49.090159  5081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026f8a8e25ee44ffb5078a9893a18c54" candidate_uuid: "b46991dd328544edb5af2c36e9b1045b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0f724fb25524a798851a7890b50e9cf" is_pre_election: true
I20240426 13:13:49.090325  5081 raft_consensus.cc:2454] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b46991dd328544edb5af2c36e9b1045b in term 0.
I20240426 13:13:49.090710  4893 leader_election.cc:302] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b46991dd328544edb5af2c36e9b1045b, f0f724fb25524a798851a7890b50e9cf; no voters: 
I20240426 13:13:49.090844  5129 raft_consensus.cc:2781] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:13:49.090893  5129 raft_consensus.cc:482] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:13:49.090938  5129 raft_consensus.cc:3037] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:49.091717  5129 raft_consensus.cc:504] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:49.092007  5129 leader_election.cc:288] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [CANDIDATE]: Term 1 election: Requested vote from peers d646fe009ae84b7d9e47e438176315d9 (127.0.99.130:33045), f0f724fb25524a798851a7890b50e9cf (127.0.99.131:41477)
I20240426 13:13:49.092343  5006 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026f8a8e25ee44ffb5078a9893a18c54" candidate_uuid: "b46991dd328544edb5af2c36e9b1045b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d646fe009ae84b7d9e47e438176315d9"
I20240426 13:13:49.092384  5081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "026f8a8e25ee44ffb5078a9893a18c54" candidate_uuid: "b46991dd328544edb5af2c36e9b1045b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0f724fb25524a798851a7890b50e9cf"
I20240426 13:13:49.092456  5081 raft_consensus.cc:3037] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:49.092458  5006 raft_consensus.cc:3037] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:49.093278  5081 raft_consensus.cc:2454] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b46991dd328544edb5af2c36e9b1045b in term 1.
I20240426 13:13:49.093312  5006 raft_consensus.cc:2454] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b46991dd328544edb5af2c36e9b1045b in term 1.
I20240426 13:13:49.093505  4893 leader_election.cc:302] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b46991dd328544edb5af2c36e9b1045b, f0f724fb25524a798851a7890b50e9cf; no voters: 
I20240426 13:13:49.093675  5129 raft_consensus.cc:2781] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:49.093916  5129 raft_consensus.cc:686] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 1 LEADER]: Becoming Leader. State: Replica: b46991dd328544edb5af2c36e9b1045b, State: Running, Role: LEADER
I20240426 13:13:49.094081  5129 consensus_queue.cc:237] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } }
I20240426 13:13:49.095441  4827 catalog_manager.cc:5484] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b reported cstate change: term changed from 0 to 1, leader changed from <none> to b46991dd328544edb5af2c36e9b1045b (127.0.99.129). New cstate: current_term: 1 leader_uuid: "b46991dd328544edb5af2c36e9b1045b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b46991dd328544edb5af2c36e9b1045b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45357 } health_report { overall_health: HEALTHY } } }
I20240426 13:13:49.142906  5081 raft_consensus.cc:1261] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 1 FOLLOWER]: Refusing update from remote peer b46991dd328544edb5af2c36e9b1045b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:49.142947  5006 raft_consensus.cc:1261] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 1 FOLLOWER]: Refusing update from remote peer b46991dd328544edb5af2c36e9b1045b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:49.143290  5132 consensus_queue.cc:1035] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0f724fb25524a798851a7890b50e9cf" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:49.143404  5129 consensus_queue.cc:1035] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d646fe009ae84b7d9e47e438176315d9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:59.142752   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:13:59.147205   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:59.147393   398 tablet_replica.cc:330] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b: stopping tablet replica
I20240426 13:13:59.147503   398 raft_consensus.cc:2229] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:13:59.147673   398 raft_consensus.cc:2258] T 026f8a8e25ee44ffb5078a9893a18c54 P b46991dd328544edb5af2c36e9b1045b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:59.163815   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:13:59.165414   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:13:59.170231   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:59.170356   398 tablet_replica.cc:330] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9: stopping tablet replica
I20240426 13:13:59.170451   398 raft_consensus.cc:2229] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:59.170531   398 raft_consensus.cc:2258] T 026f8a8e25ee44ffb5078a9893a18c54 P d646fe009ae84b7d9e47e438176315d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:59.175874   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:13:59.177222   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:13:59.180527   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:13:59.180678   398 tablet_replica.cc:330] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf: stopping tablet replica
I20240426 13:13:59.180748   398 raft_consensus.cc:2229] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:13:59.180836   398 raft_consensus.cc:2258] T 026f8a8e25ee44ffb5078a9893a18c54 P f0f724fb25524a798851a7890b50e9cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:59.196506   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:13:59.197863   398 master.cc:537] Master@127.0.99.190:33059 shutting down...
I20240426 13:13:59.200546   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:13:59.200665   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:13:59.200721   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P f89abcb265754c5b9f4472bb443f80d3: stopping tablet replica
I20240426 13:13:59.212460   398 master.cc:552] Master@127.0.99.190:33059 shutdown complete.
[       OK ] TxnParticipantITest.TestProxyTabletNotFound (10458 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientGetMetadata
I20240426 13:13:59.220342   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:34871
I20240426 13:13:59.220613   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:59.222355  5186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:59.223431   398 server_base.cc:1034] running on GCE node
W20240426 13:13:59.223915  5189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:59.224673  5187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:59.224862   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:59.224907   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:59.224928   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137239224927 us; error 0 us; skew 500 ppm
I20240426 13:13:59.225713   398 webserver.cc:460] Webserver started at http://127.0.99.190:34921/ using document root <none> and password file <none>
I20240426 13:13:59.225843   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:59.225900   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:59.225996   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:59.226317   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "94a8269d88804dadaa74619174c966cc"
format_stamp: "Formatted at 2024-04-26 13:13:59 on dist-test-slave-2jsl"
I20240426 13:13:59.227488   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:59.228224  5194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.228407   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:13:59.228462   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "94a8269d88804dadaa74619174c966cc"
format_stamp: "Formatted at 2024-04-26 13:13:59 on dist-test-slave-2jsl"
I20240426 13:13:59.228521   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:59.251024   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:59.251255   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:59.254588   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:34871
I20240426 13:13:59.261243  5246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:34871 every 8 connection(s)
I20240426 13:13:59.261564  5247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:59.263044  5247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc: Bootstrap starting.
I20240426 13:13:59.263612  5247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:59.264302  5247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc: No bootstrap required, opened a new log
I20240426 13:13:59.264549  5247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a8269d88804dadaa74619174c966cc" member_type: VOTER }
I20240426 13:13:59.264622  5247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:59.264648  5247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94a8269d88804dadaa74619174c966cc, State: Initialized, Role: FOLLOWER
I20240426 13:13:59.264753  5247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a8269d88804dadaa74619174c966cc" member_type: VOTER }
I20240426 13:13:59.264834  5247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:13:59.264865  5247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:13:59.264899  5247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:59.265447  5247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a8269d88804dadaa74619174c966cc" member_type: VOTER }
I20240426 13:13:59.265544  5247 leader_election.cc:302] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94a8269d88804dadaa74619174c966cc; no voters: 
I20240426 13:13:59.265676  5247 leader_election.cc:288] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:13:59.265760  5250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:59.265970  5250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 1 LEADER]: Becoming Leader. State: Replica: 94a8269d88804dadaa74619174c966cc, State: Running, Role: LEADER
I20240426 13:13:59.266069  5247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:13:59.266094  5250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a8269d88804dadaa74619174c966cc" member_type: VOTER }
I20240426 13:13:59.266513  5251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94a8269d88804dadaa74619174c966cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a8269d88804dadaa74619174c966cc" member_type: VOTER } }
I20240426 13:13:59.266551  5252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94a8269d88804dadaa74619174c966cc. Latest consensus state: current_term: 1 leader_uuid: "94a8269d88804dadaa74619174c966cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94a8269d88804dadaa74619174c966cc" member_type: VOTER } }
I20240426 13:13:59.266685  5252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:59.266904  5251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [sys.catalog]: This master's current role is: LEADER
I20240426 13:13:59.267127  5258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:13:59.267958  5258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:13:59.268123   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:13:59.269451  5258 catalog_manager.cc:1347] Generated new cluster ID: 617eae64bb5e4bcab9ecc55c8f78d324
I20240426 13:13:59.269500  5258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:13:59.282797  5258 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:13:59.283181  5258 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:13:59.287983  5258 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc: Generated new TSK 0
I20240426 13:13:59.288092  5258 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:13:59.300091   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:59.301702  5268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:59.302210  5269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:59.302980  5271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:59.303798   398 server_base.cc:1034] running on GCE node
I20240426 13:13:59.303915   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:59.303946   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:59.303958   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137239303959 us; error 0 us; skew 500 ppm
I20240426 13:13:59.304678   398 webserver.cc:460] Webserver started at http://127.0.99.129:41459/ using document root <none> and password file <none>
I20240426 13:13:59.304807   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:59.304843   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:59.304895   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:59.305182   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "a3e22dbb53aa4463bb49a8be5e4285ed"
format_stamp: "Formatted at 2024-04-26 13:13:59 on dist-test-slave-2jsl"
I20240426 13:13:59.306500   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:13:59.307214  5276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.307446   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:59.307525   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "a3e22dbb53aa4463bb49a8be5e4285ed"
format_stamp: "Formatted at 2024-04-26 13:13:59 on dist-test-slave-2jsl"
I20240426 13:13:59.307593   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:59.318672   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:59.318934   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:59.319130   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:59.319469   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:59.319504   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.319542   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:59.319579   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.323062   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:45113
I20240426 13:13:59.323079  5339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:45113 every 8 connection(s)
I20240426 13:13:59.323791   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:59.326078  5345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:59.326205  5344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:59.326771  5347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:59.327236  5340 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34871
I20240426 13:13:59.327311  5340 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:59.327468  5340 heartbeater.cc:511] Master 127.0.99.190:34871 requested a full tablet report, sending...
I20240426 13:13:59.327585   398 server_base.cc:1034] running on GCE node
I20240426 13:13:59.327756   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:59.327800   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:59.327822   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137239327821 us; error 0 us; skew 500 ppm
I20240426 13:13:59.327963  5211 ts_manager.cc:194] Registered new tserver with Master: a3e22dbb53aa4463bb49a8be5e4285ed (127.0.99.129:45113)
I20240426 13:13:59.328579  5211 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47100
I20240426 13:13:59.328615   398 webserver.cc:460] Webserver started at http://127.0.99.130:41833/ using document root <none> and password file <none>
I20240426 13:13:59.328704   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:59.328729   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:59.328771   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:59.329056   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "46bc1225d4df47b9aac65e806a6f59e1"
format_stamp: "Formatted at 2024-04-26 13:13:59 on dist-test-slave-2jsl"
I20240426 13:13:59.330148   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:59.330825  5352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.331077   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:59.331135   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "46bc1225d4df47b9aac65e806a6f59e1"
format_stamp: "Formatted at 2024-04-26 13:13:59 on dist-test-slave-2jsl"
I20240426 13:13:59.331185   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:59.340992   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:59.341224   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:59.341450   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:59.341794   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:59.341828   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.341871   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:59.341898   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.345676   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:33079
I20240426 13:13:59.345695  5415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:33079 every 8 connection(s)
I20240426 13:13:59.346393   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:13:59.348073  5419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:59.348595  5420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:13:59.349344  5422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:13:59.349946  5416 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34871
I20240426 13:13:59.350042  5416 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:59.350076   398 server_base.cc:1034] running on GCE node
I20240426 13:13:59.350175   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:13:59.350203   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:13:59.350219   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137239350219 us; error 0 us; skew 500 ppm
I20240426 13:13:59.350219  5416 heartbeater.cc:511] Master 127.0.99.190:34871 requested a full tablet report, sending...
I20240426 13:13:59.350821  5211 ts_manager.cc:194] Registered new tserver with Master: 46bc1225d4df47b9aac65e806a6f59e1 (127.0.99.130:33079)
I20240426 13:13:59.351073   398 webserver.cc:460] Webserver started at http://127.0.99.131:39323/ using document root <none> and password file <none>
I20240426 13:13:59.351197   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:13:59.351248   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:13:59.351286  5211 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47104
I20240426 13:13:59.351307   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:13:59.351694   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "e97e85fd821a4b149502bb0ed2d89c7d"
format_stamp: "Formatted at 2024-04-26 13:13:59 on dist-test-slave-2jsl"
I20240426 13:13:59.352902   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:59.353561  5427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.353744   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.353802   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "e97e85fd821a4b149502bb0ed2d89c7d"
format_stamp: "Formatted at 2024-04-26 13:13:59 on dist-test-slave-2jsl"
I20240426 13:13:59.353863   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientGetMetadata.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:13:59.381503   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:13:59.381770   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:13:59.382014   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:13:59.382392   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:13:59.382431   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.382470   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:13:59.382504   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:13:59.386708   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:34791
I20240426 13:13:59.386727  5490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:34791 every 8 connection(s)
I20240426 13:13:59.390048  5491 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34871
I20240426 13:13:59.390112  5491 heartbeater.cc:464] Registering TS with master...
I20240426 13:13:59.390251  5491 heartbeater.cc:511] Master 127.0.99.190:34871 requested a full tablet report, sending...
I20240426 13:13:59.390733  5211 ts_manager.cc:194] Registered new tserver with Master: e97e85fd821a4b149502bb0ed2d89c7d (127.0.99.131:34791)
I20240426 13:13:59.391165  5211 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47112
I20240426 13:13:59.391621   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004589352s
I20240426 13:13:59.392037   398 test_util.cc:268] Using random seed: -1388310748
I20240426 13:13:59.398214  5211 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47126:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:13:59.398497  5211 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:13:59.406688  5380 tablet_service.cc:1467] Processing CreateTablet for tablet ce4f4c5e55a04bd09940fd81f39868af (DEFAULT_TABLE table=test-workload [id=92521d81f3e947eaa447999650b19eeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:59.406956  5380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4f4c5e55a04bd09940fd81f39868af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:59.407352  5304 tablet_service.cc:1467] Processing CreateTablet for tablet ce4f4c5e55a04bd09940fd81f39868af (DEFAULT_TABLE table=test-workload [id=92521d81f3e947eaa447999650b19eeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:59.407570  5304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4f4c5e55a04bd09940fd81f39868af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:59.409132  5506 tablet_bootstrap.cc:492] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1: Bootstrap starting.
I20240426 13:13:59.409547  5507 tablet_bootstrap.cc:492] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed: Bootstrap starting.
I20240426 13:13:59.409921  5455 tablet_service.cc:1467] Processing CreateTablet for tablet ce4f4c5e55a04bd09940fd81f39868af (DEFAULT_TABLE table=test-workload [id=92521d81f3e947eaa447999650b19eeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:13:59.410156  5506 tablet_bootstrap.cc:654] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:59.410190  5455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce4f4c5e55a04bd09940fd81f39868af. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:13:59.410470  5507 tablet_bootstrap.cc:654] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:59.411259  5506 tablet_bootstrap.cc:492] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1: No bootstrap required, opened a new log
I20240426 13:13:59.411330  5506 ts_tablet_manager.cc:1367] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:13:59.411506  5507 tablet_bootstrap.cc:492] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed: No bootstrap required, opened a new log
I20240426 13:13:59.411566  5507 ts_tablet_manager.cc:1367] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed: Time spent bootstrapping tablet: real 0.002s	user 0.000s	sys 0.001s
I20240426 13:13:59.411689  5506 raft_consensus.cc:348] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.411773  5506 raft_consensus.cc:374] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:59.411803  5506 raft_consensus.cc:729] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46bc1225d4df47b9aac65e806a6f59e1, State: Initialized, Role: FOLLOWER
I20240426 13:13:59.411832  5507 raft_consensus.cc:348] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.411926  5507 raft_consensus.cc:374] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:59.411962  5507 raft_consensus.cc:729] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3e22dbb53aa4463bb49a8be5e4285ed, State: Initialized, Role: FOLLOWER
I20240426 13:13:59.411911  5506 consensus_queue.cc:260] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.412062  5507 consensus_queue.cc:260] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.412253  5506 ts_tablet_manager.cc:1398] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:59.412333  5511 tablet_bootstrap.cc:492] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d: Bootstrap starting.
I20240426 13:13:59.412395  5507 ts_tablet_manager.cc:1398] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:13:59.412405  5416 heartbeater.cc:503] Master 127.0.99.190:34871 was elected leader, sending a full tablet report...
I20240426 13:13:59.412606  5340 heartbeater.cc:503] Master 127.0.99.190:34871 was elected leader, sending a full tablet report...
I20240426 13:13:59.413760  5511 tablet_bootstrap.cc:654] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d: Neither blocks nor log segments found. Creating new log.
I20240426 13:13:59.415381  5511 tablet_bootstrap.cc:492] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d: No bootstrap required, opened a new log
I20240426 13:13:59.415458  5511 ts_tablet_manager.cc:1367] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:13:59.415773  5511 raft_consensus.cc:348] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.415850  5511 raft_consensus.cc:374] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:13:59.415874  5511 raft_consensus.cc:729] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97e85fd821a4b149502bb0ed2d89c7d, State: Initialized, Role: FOLLOWER
I20240426 13:13:59.416013  5511 consensus_queue.cc:260] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.416266  5511 ts_tablet_manager.cc:1398] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:13:59.416330  5491 heartbeater.cc:503] Master 127.0.99.190:34871 was elected leader, sending a full tablet report...
I20240426 13:13:59.602777  5512 raft_consensus.cc:482] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:13:59.602890  5512 raft_consensus.cc:504] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.603358  5512 leader_election.cc:288] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46bc1225d4df47b9aac65e806a6f59e1 (127.0.99.130:33079), e97e85fd821a4b149502bb0ed2d89c7d (127.0.99.131:34791)
I20240426 13:13:59.606956  5465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4f4c5e55a04bd09940fd81f39868af" candidate_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" is_pre_election: true
I20240426 13:13:59.607069  5465 raft_consensus.cc:2454] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3e22dbb53aa4463bb49a8be5e4285ed in term 0.
I20240426 13:13:59.607306  5390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4f4c5e55a04bd09940fd81f39868af" candidate_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46bc1225d4df47b9aac65e806a6f59e1" is_pre_election: true
I20240426 13:13:59.607355  5277 leader_election.cc:302] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e22dbb53aa4463bb49a8be5e4285ed, e97e85fd821a4b149502bb0ed2d89c7d; no voters: 
I20240426 13:13:59.607398  5390 raft_consensus.cc:2454] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3e22dbb53aa4463bb49a8be5e4285ed in term 0.
I20240426 13:13:59.607527  5512 raft_consensus.cc:2781] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:13:59.607587  5512 raft_consensus.cc:482] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:13:59.607659  5512 raft_consensus.cc:3037] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:59.608453  5512 raft_consensus.cc:504] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.608747  5512 leader_election.cc:288] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [CANDIDATE]: Term 1 election: Requested vote from peers 46bc1225d4df47b9aac65e806a6f59e1 (127.0.99.130:33079), e97e85fd821a4b149502bb0ed2d89c7d (127.0.99.131:34791)
I20240426 13:13:59.608983  5390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4f4c5e55a04bd09940fd81f39868af" candidate_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46bc1225d4df47b9aac65e806a6f59e1"
I20240426 13:13:59.609023  5465 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce4f4c5e55a04bd09940fd81f39868af" candidate_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e97e85fd821a4b149502bb0ed2d89c7d"
I20240426 13:13:59.609061  5390 raft_consensus.cc:3037] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:59.609123  5465 raft_consensus.cc:3037] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:13:59.609848  5465 raft_consensus.cc:2454] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3e22dbb53aa4463bb49a8be5e4285ed in term 1.
I20240426 13:13:59.609854  5390 raft_consensus.cc:2454] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3e22dbb53aa4463bb49a8be5e4285ed in term 1.
I20240426 13:13:59.610078  5277 leader_election.cc:302] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3e22dbb53aa4463bb49a8be5e4285ed, e97e85fd821a4b149502bb0ed2d89c7d; no voters: 
I20240426 13:13:59.610242  5512 raft_consensus.cc:2781] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:13:59.610388  5512 raft_consensus.cc:686] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 1 LEADER]: Becoming Leader. State: Replica: a3e22dbb53aa4463bb49a8be5e4285ed, State: Running, Role: LEADER
I20240426 13:13:59.610512  5512 consensus_queue.cc:237] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } }
I20240426 13:13:59.611711  5211 catalog_manager.cc:5484] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed reported cstate change: term changed from 0 to 1, leader changed from <none> to a3e22dbb53aa4463bb49a8be5e4285ed (127.0.99.129). New cstate: current_term: 1 leader_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3e22dbb53aa4463bb49a8be5e4285ed" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 } health_report { overall_health: UNKNOWN } } }
I20240426 13:13:59.651000  5465 raft_consensus.cc:1261] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 1 FOLLOWER]: Refusing update from remote peer a3e22dbb53aa4463bb49a8be5e4285ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:59.651001  5390 raft_consensus.cc:1261] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 1 FOLLOWER]: Refusing update from remote peer a3e22dbb53aa4463bb49a8be5e4285ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:13:59.651394  5516 consensus_queue.cc:1035] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "46bc1225d4df47b9aac65e806a6f59e1" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 33079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:13:59.651587  5512 consensus_queue.cc:1035] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "e97e85fd821a4b149502bb0ed2d89c7d" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 34791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:09.681614   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:14:09.685499   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:09.685644   398 tablet_replica.cc:330] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed: stopping tablet replica
I20240426 13:14:09.685747   398 raft_consensus.cc:2229] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:09.685918   398 raft_consensus.cc:2258] T ce4f4c5e55a04bd09940fd81f39868af P a3e22dbb53aa4463bb49a8be5e4285ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:09.701604   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:14:09.703100   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:14:09.706724   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:09.706859   398 tablet_replica.cc:330] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1: stopping tablet replica
I20240426 13:14:09.706959   398 raft_consensus.cc:2229] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:09.707062   398 raft_consensus.cc:2258] T ce4f4c5e55a04bd09940fd81f39868af P 46bc1225d4df47b9aac65e806a6f59e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:09.713858   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:14:09.715103   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:14:09.718385   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:09.718569   398 tablet_replica.cc:330] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d: stopping tablet replica
I20240426 13:14:09.718685   398 raft_consensus.cc:2229] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:09.718780   398 raft_consensus.cc:2258] T ce4f4c5e55a04bd09940fd81f39868af P e97e85fd821a4b149502bb0ed2d89c7d [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:09.733955   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:14:09.735358   398 master.cc:537] Master@127.0.99.190:34871 shutting down...
I20240426 13:14:09.738343   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:09.738458   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:09.738514   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 94a8269d88804dadaa74619174c966cc: stopping tablet replica
I20240426 13:14:09.750466   398 master.cc:552] Master@127.0.99.190:34871 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientGetMetadata (10538 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock
I20240426 13:14:09.759172   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:43095
I20240426 13:14:09.759492   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:09.762125  5574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:09.762637  5576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:09.762842  5573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:09.763782   398 server_base.cc:1034] running on GCE node
I20240426 13:14:09.763914   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:09.763947   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:09.763959   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137249763959 us; error 0 us; skew 500 ppm
I20240426 13:14:09.764744   398 webserver.cc:460] Webserver started at http://127.0.99.190:33831/ using document root <none> and password file <none>
I20240426 13:14:09.764875   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:09.764926   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:09.764990   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:09.765332   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "028166e4b4144ff7adaed84c3abaac83"
format_stamp: "Formatted at 2024-04-26 13:14:09 on dist-test-slave-2jsl"
I20240426 13:14:09.766721   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:14:09.767472  5581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.767704   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:09.767757   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "028166e4b4144ff7adaed84c3abaac83"
format_stamp: "Formatted at 2024-04-26 13:14:09 on dist-test-slave-2jsl"
I20240426 13:14:09.767804   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:09.782101   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:09.782307   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:09.786278   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:43095
I20240426 13:14:09.791153  5633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:43095 every 8 connection(s)
I20240426 13:14:09.791513  5634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:09.792968  5634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83: Bootstrap starting.
I20240426 13:14:09.793555  5634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:09.794416  5634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83: No bootstrap required, opened a new log
I20240426 13:14:09.794834  5634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028166e4b4144ff7adaed84c3abaac83" member_type: VOTER }
I20240426 13:14:09.794922  5634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:09.794945  5634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 028166e4b4144ff7adaed84c3abaac83, State: Initialized, Role: FOLLOWER
I20240426 13:14:09.795078  5634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028166e4b4144ff7adaed84c3abaac83" member_type: VOTER }
I20240426 13:14:09.795173  5634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:14:09.795205  5634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:14:09.795241  5634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:09.795831  5634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028166e4b4144ff7adaed84c3abaac83" member_type: VOTER }
I20240426 13:14:09.795933  5634 leader_election.cc:302] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 028166e4b4144ff7adaed84c3abaac83; no voters: 
I20240426 13:14:09.796067  5634 leader_election.cc:288] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:14:09.796162  5637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:09.796373  5637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 1 LEADER]: Becoming Leader. State: Replica: 028166e4b4144ff7adaed84c3abaac83, State: Running, Role: LEADER
I20240426 13:14:09.796460  5634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:14:09.796504  5637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028166e4b4144ff7adaed84c3abaac83" member_type: VOTER }
I20240426 13:14:09.796875  5638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "028166e4b4144ff7adaed84c3abaac83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028166e4b4144ff7adaed84c3abaac83" member_type: VOTER } }
I20240426 13:14:09.796897  5639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 028166e4b4144ff7adaed84c3abaac83. Latest consensus state: current_term: 1 leader_uuid: "028166e4b4144ff7adaed84c3abaac83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "028166e4b4144ff7adaed84c3abaac83" member_type: VOTER } }
I20240426 13:14:09.797080  5639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:09.796999  5638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:09.797508  5644 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:14:09.798403  5644 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:14:09.798622   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:14:09.799952  5644 catalog_manager.cc:1347] Generated new cluster ID: 8260f650e31d4eed805f4cc79d32badd
I20240426 13:14:09.799996  5644 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:14:09.809367  5644 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:14:09.809830  5644 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:14:09.819168  5644 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83: Generated new TSK 0
I20240426 13:14:09.819308  5644 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:14:09.830691   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:09.832540  5655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:09.833067  5656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:09.833942  5658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:09.834758   398 server_base.cc:1034] running on GCE node
I20240426 13:14:09.834883   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:09.834918   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:09.834937   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137249834937 us; error 0 us; skew 500 ppm
I20240426 13:14:09.835737   398 webserver.cc:460] Webserver started at http://127.0.99.129:43981/ using document root <none> and password file <none>
I20240426 13:14:09.835889   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:09.835937   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:09.836007   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:09.836365   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "e8dc07800fa24910b128aa61c575106a"
format_stamp: "Formatted at 2024-04-26 13:14:09 on dist-test-slave-2jsl"
I20240426 13:14:09.837679   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:14:09.838474  5663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.838649   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.838717   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "e8dc07800fa24910b128aa61c575106a"
format_stamp: "Formatted at 2024-04-26 13:14:09 on dist-test-slave-2jsl"
I20240426 13:14:09.838775   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:09.856443   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:09.856690   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:09.856917   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:09.857282   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:09.857319   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.857362   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:09.857391   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.860924   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:41611
I20240426 13:14:09.861742  5726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:41611 every 8 connection(s)
I20240426 13:14:09.861850   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:09.863377  5731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:09.863888  5732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:09.864666  5734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:09.865480   398 server_base.cc:1034] running on GCE node
I20240426 13:14:09.865660   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:09.865734   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:09.865761   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137249865761 us; error 0 us; skew 500 ppm
I20240426 13:14:09.866264  5727 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43095
I20240426 13:14:09.866351  5727 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:09.866619  5727 heartbeater.cc:511] Master 127.0.99.190:43095 requested a full tablet report, sending...
I20240426 13:14:09.866757   398 webserver.cc:460] Webserver started at http://127.0.99.130:38973/ using document root <none> and password file <none>
I20240426 13:14:09.866895   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:09.866945   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:09.867036   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:09.867326  5598 ts_manager.cc:194] Registered new tserver with Master: e8dc07800fa24910b128aa61c575106a (127.0.99.129:41611)
I20240426 13:14:09.867455   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "6924b5640996454ca4eb21f5cf225b50"
format_stamp: "Formatted at 2024-04-26 13:14:09 on dist-test-slave-2jsl"
I20240426 13:14:09.868276  5598 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53556
I20240426 13:14:09.868947   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:09.869760  5739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.870035   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:09.870107   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "6924b5640996454ca4eb21f5cf225b50"
format_stamp: "Formatted at 2024-04-26 13:14:09 on dist-test-slave-2jsl"
I20240426 13:14:09.870173   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:09.878270   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:09.878506   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:09.878723   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:09.879048   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:09.879083   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.879123   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:09.879153   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.882565   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:39093
I20240426 13:14:09.882587  5802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:39093 every 8 connection(s)
I20240426 13:14:09.883208   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:09.884634  5806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:09.885128  5807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:09.885989  5809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:09.886819   398 server_base.cc:1034] running on GCE node
I20240426 13:14:09.886976   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:09.887058   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:09.887077   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137249887077 us; error 0 us; skew 500 ppm
I20240426 13:14:09.887485  5803 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43095
I20240426 13:14:09.887560  5803 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:09.887773  5803 heartbeater.cc:511] Master 127.0.99.190:43095 requested a full tablet report, sending...
I20240426 13:14:09.888000   398 webserver.cc:460] Webserver started at http://127.0.99.131:38459/ using document root <none> and password file <none>
I20240426 13:14:09.888099   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:09.888136   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:09.888192   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:09.888327  5598 ts_manager.cc:194] Registered new tserver with Master: 6924b5640996454ca4eb21f5cf225b50 (127.0.99.130:39093)
I20240426 13:14:09.888631   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "a4fb3f9d5cd048ef84420735ede354f2"
format_stamp: "Formatted at 2024-04-26 13:14:09 on dist-test-slave-2jsl"
I20240426 13:14:09.888973  5598 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53570
I20240426 13:14:09.890147   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:09.890892  5814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.891083   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:09.891135   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "a4fb3f9d5cd048ef84420735ede354f2"
format_stamp: "Formatted at 2024-04-26 13:14:09 on dist-test-slave-2jsl"
I20240426 13:14:09.891180   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:09.907492   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:09.907686   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:09.907856   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:09.908190   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:09.908221   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.908246   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:09.908277   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:09.911929   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:41233
I20240426 13:14:09.911952  5877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:41233 every 8 connection(s)
I20240426 13:14:09.915423  5878 heartbeater.cc:347] Connected to a master server at 127.0.99.190:43095
I20240426 13:14:09.915501  5878 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:09.915648  5878 heartbeater.cc:511] Master 127.0.99.190:43095 requested a full tablet report, sending...
I20240426 13:14:09.916134  5598 ts_manager.cc:194] Registered new tserver with Master: a4fb3f9d5cd048ef84420735ede354f2 (127.0.99.131:41233)
I20240426 13:14:09.916594  5598 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53586
I20240426 13:14:09.916734   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004502166s
I20240426 13:14:09.917208   398 test_util.cc:268] Using random seed: -1377785577
I20240426 13:14:09.923499  5598 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53602:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:14:09.923883  5598 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:14:09.932160  5767 tablet_service.cc:1467] Processing CreateTablet for tablet fc66884f9a2443bf9bc888dd10c25a42 (DEFAULT_TABLE table=test-workload [id=9749817e2d424ce4835ae5fe7b6a826a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:09.932422  5767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc66884f9a2443bf9bc888dd10c25a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:09.932576  5691 tablet_service.cc:1467] Processing CreateTablet for tablet fc66884f9a2443bf9bc888dd10c25a42 (DEFAULT_TABLE table=test-workload [id=9749817e2d424ce4835ae5fe7b6a826a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:09.932744  5842 tablet_service.cc:1467] Processing CreateTablet for tablet fc66884f9a2443bf9bc888dd10c25a42 (DEFAULT_TABLE table=test-workload [id=9749817e2d424ce4835ae5fe7b6a826a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:09.932788  5691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc66884f9a2443bf9bc888dd10c25a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:09.932934  5842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc66884f9a2443bf9bc888dd10c25a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:09.934636  5893 tablet_bootstrap.cc:492] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50: Bootstrap starting.
I20240426 13:14:09.935142  5895 tablet_bootstrap.cc:492] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a: Bootstrap starting.
I20240426 13:14:09.935454  5893 tablet_bootstrap.cc:654] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:09.936015  5895 tablet_bootstrap.cc:654] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:09.936373  5893 tablet_bootstrap.cc:492] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50: No bootstrap required, opened a new log
I20240426 13:14:09.936476  5893 ts_tablet_manager.cc:1367] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:14:09.936584  5894 tablet_bootstrap.cc:492] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2: Bootstrap starting.
I20240426 13:14:09.936868  5893 raft_consensus.cc:348] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:09.936975  5893 raft_consensus.cc:374] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:09.937022  5893 raft_consensus.cc:729] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6924b5640996454ca4eb21f5cf225b50, State: Initialized, Role: FOLLOWER
I20240426 13:14:09.937106  5895 tablet_bootstrap.cc:492] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a: No bootstrap required, opened a new log
I20240426 13:14:09.937179  5895 ts_tablet_manager.cc:1367] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:14:09.937162  5893 consensus_queue.cc:260] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:09.937405  5894 tablet_bootstrap.cc:654] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:09.937515  5893 ts_tablet_manager.cc:1398] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:09.937578  5803 heartbeater.cc:503] Master 127.0.99.190:43095 was elected leader, sending a full tablet report...
I20240426 13:14:09.937551  5895 raft_consensus.cc:348] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:09.937659  5895 raft_consensus.cc:374] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:09.937702  5895 raft_consensus.cc:729] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8dc07800fa24910b128aa61c575106a, State: Initialized, Role: FOLLOWER
I20240426 13:14:09.937824  5895 consensus_queue.cc:260] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:09.938125  5895 ts_tablet_manager.cc:1398] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:09.938251  5727 heartbeater.cc:503] Master 127.0.99.190:43095 was elected leader, sending a full tablet report...
I20240426 13:14:09.938977  5894 tablet_bootstrap.cc:492] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2: No bootstrap required, opened a new log
I20240426 13:14:09.939047  5894 ts_tablet_manager.cc:1367] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:14:09.939359  5894 raft_consensus.cc:348] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:09.939442  5894 raft_consensus.cc:374] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:09.939477  5894 raft_consensus.cc:729] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fb3f9d5cd048ef84420735ede354f2, State: Initialized, Role: FOLLOWER
I20240426 13:14:09.939574  5894 consensus_queue.cc:260] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:09.939894  5894 ts_tablet_manager.cc:1398] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:09.939915  5878 heartbeater.cc:503] Master 127.0.99.190:43095 was elected leader, sending a full tablet report...
I20240426 13:14:10.053339  5899 raft_consensus.cc:482] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:14:10.053458  5899 raft_consensus.cc:504] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:10.053948  5899 leader_election.cc:288] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6924b5640996454ca4eb21f5cf225b50 (127.0.99.130:39093), a4fb3f9d5cd048ef84420735ede354f2 (127.0.99.131:41233)
I20240426 13:14:10.057734  5777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc66884f9a2443bf9bc888dd10c25a42" candidate_uuid: "e8dc07800fa24910b128aa61c575106a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6924b5640996454ca4eb21f5cf225b50" is_pre_election: true
I20240426 13:14:10.057739  5852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc66884f9a2443bf9bc888dd10c25a42" candidate_uuid: "e8dc07800fa24910b128aa61c575106a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fb3f9d5cd048ef84420735ede354f2" is_pre_election: true
I20240426 13:14:10.057868  5777 raft_consensus.cc:2454] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8dc07800fa24910b128aa61c575106a in term 0.
I20240426 13:14:10.058038  5852 raft_consensus.cc:2454] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8dc07800fa24910b128aa61c575106a in term 0.
I20240426 13:14:10.058313  5664 leader_election.cc:302] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6924b5640996454ca4eb21f5cf225b50, e8dc07800fa24910b128aa61c575106a; no voters: 
I20240426 13:14:10.058470  5899 raft_consensus.cc:2781] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:14:10.058531  5899 raft_consensus.cc:482] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:14:10.058583  5899 raft_consensus.cc:3037] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:10.059345  5899 raft_consensus.cc:504] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:10.059631  5899 leader_election.cc:288] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [CANDIDATE]: Term 1 election: Requested vote from peers 6924b5640996454ca4eb21f5cf225b50 (127.0.99.130:39093), a4fb3f9d5cd048ef84420735ede354f2 (127.0.99.131:41233)
I20240426 13:14:10.059864  5777 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc66884f9a2443bf9bc888dd10c25a42" candidate_uuid: "e8dc07800fa24910b128aa61c575106a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6924b5640996454ca4eb21f5cf225b50"
I20240426 13:14:10.059899  5852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc66884f9a2443bf9bc888dd10c25a42" candidate_uuid: "e8dc07800fa24910b128aa61c575106a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fb3f9d5cd048ef84420735ede354f2"
I20240426 13:14:10.059938  5777 raft_consensus.cc:3037] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:10.059986  5852 raft_consensus.cc:3037] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:10.060736  5777 raft_consensus.cc:2454] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8dc07800fa24910b128aa61c575106a in term 1.
I20240426 13:14:10.060916  5852 raft_consensus.cc:2454] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8dc07800fa24910b128aa61c575106a in term 1.
I20240426 13:14:10.060986  5664 leader_election.cc:302] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6924b5640996454ca4eb21f5cf225b50, e8dc07800fa24910b128aa61c575106a; no voters: 
I20240426 13:14:10.061138  5899 raft_consensus.cc:2781] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:10.061342  5899 raft_consensus.cc:686] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 1 LEADER]: Becoming Leader. State: Replica: e8dc07800fa24910b128aa61c575106a, State: Running, Role: LEADER
I20240426 13:14:10.061502  5899 consensus_queue.cc:237] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } }
I20240426 13:14:10.062990  5598 catalog_manager.cc:5484] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a reported cstate change: term changed from 0 to 1, leader changed from <none> to e8dc07800fa24910b128aa61c575106a (127.0.99.129). New cstate: current_term: 1 leader_uuid: "e8dc07800fa24910b128aa61c575106a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8dc07800fa24910b128aa61c575106a" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 41611 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 } health_report { overall_health: UNKNOWN } } }
I20240426 13:14:10.097796  5852 raft_consensus.cc:1261] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 1 FOLLOWER]: Refusing update from remote peer e8dc07800fa24910b128aa61c575106a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:10.097831  5777 raft_consensus.cc:1261] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 1 FOLLOWER]: Refusing update from remote peer e8dc07800fa24910b128aa61c575106a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:10.098157  5899 consensus_queue.cc:1035] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fb3f9d5cd048ef84420735ede354f2" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:10.098342  5903 consensus_queue.cc:1035] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6924b5640996454ca4eb21f5cf225b50" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39093 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:10.107043  5913 mvcc.cc:205] Tried to move back new op lower bound from 7021106176400920576 to 7021106176253308928. Current Snapshot: MvccSnapshot[applied={T|T < 7021106176397627392}]
I20240426 13:14:20.118878   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:14:20.123345   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:20.123476   398 tablet_replica.cc:330] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a: stopping tablet replica
I20240426 13:14:20.123564   398 raft_consensus.cc:2229] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:20.123692   398 raft_consensus.cc:2258] T fc66884f9a2443bf9bc888dd10c25a42 P e8dc07800fa24910b128aa61c575106a [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:20.139405   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:14:20.140949   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:14:20.144876   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:20.145011   398 tablet_replica.cc:330] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50: stopping tablet replica
I20240426 13:14:20.145102   398 raft_consensus.cc:2229] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:20.145203   398 raft_consensus.cc:2258] T fc66884f9a2443bf9bc888dd10c25a42 P 6924b5640996454ca4eb21f5cf225b50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:20.160283   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:14:20.161594   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:14:20.165272   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:20.165395   398 tablet_replica.cc:330] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2: stopping tablet replica
I20240426 13:14:20.165470   398 raft_consensus.cc:2229] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:20.165566   398 raft_consensus.cc:2258] T fc66884f9a2443bf9bc888dd10c25a42 P a4fb3f9d5cd048ef84420735ede354f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:20.173497   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:14:20.174906   398 master.cc:537] Master@127.0.99.190:43095 shutting down...
I20240426 13:14:20.178159   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:20.178243   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:20.178280   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 028166e4b4144ff7adaed84c3abaac83: stopping tablet replica
I20240426 13:14:20.191076   398 master.cc:552] Master@127.0.99.190:43095 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientBeginTxnDoesntLock (10439 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientCommitSequence
I20240426 13:14:20.198810   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:34057
I20240426 13:14:20.199033   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:20.201018  5968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:20.201225  5967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:20.202064  5970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:20.202827   398 server_base.cc:1034] running on GCE node
I20240426 13:14:20.202936   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:20.202967   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:20.202984   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137260202984 us; error 0 us; skew 500 ppm
I20240426 13:14:20.203666   398 webserver.cc:460] Webserver started at http://127.0.99.190:38471/ using document root <none> and password file <none>
I20240426 13:14:20.203780   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:20.203820   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:20.203882   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:20.204177   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "b02e412889e1467eb0e0984f36668c79"
format_stamp: "Formatted at 2024-04-26 13:14:20 on dist-test-slave-2jsl"
I20240426 13:14:20.205381   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:20.206153  5975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.206346   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:20.206401   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "b02e412889e1467eb0e0984f36668c79"
format_stamp: "Formatted at 2024-04-26 13:14:20 on dist-test-slave-2jsl"
I20240426 13:14:20.206459   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:20.212131   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:20.212368   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:20.215708   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:34057
I20240426 13:14:20.216850  6027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:34057 every 8 connection(s)
I20240426 13:14:20.218964  6028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:20.220345  6028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79: Bootstrap starting.
I20240426 13:14:20.220891  6028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:20.221616  6028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79: No bootstrap required, opened a new log
I20240426 13:14:20.221944  6028 raft_consensus.cc:348] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02e412889e1467eb0e0984f36668c79" member_type: VOTER }
I20240426 13:14:20.222054  6028 raft_consensus.cc:374] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:20.222082  6028 raft_consensus.cc:729] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02e412889e1467eb0e0984f36668c79, State: Initialized, Role: FOLLOWER
I20240426 13:14:20.222198  6028 consensus_queue.cc:260] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02e412889e1467eb0e0984f36668c79" member_type: VOTER }
I20240426 13:14:20.222255  6028 raft_consensus.cc:388] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:14:20.222287  6028 raft_consensus.cc:482] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:14:20.222333  6028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:20.222903  6028 raft_consensus.cc:504] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02e412889e1467eb0e0984f36668c79" member_type: VOTER }
I20240426 13:14:20.223001  6028 leader_election.cc:302] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b02e412889e1467eb0e0984f36668c79; no voters: 
I20240426 13:14:20.223161  6028 leader_election.cc:288] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:14:20.223248  6031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:20.223424  6031 raft_consensus.cc:686] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 1 LEADER]: Becoming Leader. State: Replica: b02e412889e1467eb0e0984f36668c79, State: Running, Role: LEADER
I20240426 13:14:20.223482  6028 sys_catalog.cc:564] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:14:20.223553  6031 consensus_queue.cc:237] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02e412889e1467eb0e0984f36668c79" member_type: VOTER }
I20240426 13:14:20.223891  6032 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b02e412889e1467eb0e0984f36668c79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02e412889e1467eb0e0984f36668c79" member_type: VOTER } }
I20240426 13:14:20.223902  6033 sys_catalog.cc:455] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b02e412889e1467eb0e0984f36668c79. Latest consensus state: current_term: 1 leader_uuid: "b02e412889e1467eb0e0984f36668c79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b02e412889e1467eb0e0984f36668c79" member_type: VOTER } }
I20240426 13:14:20.223990  6032 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:20.223999  6033 sys_catalog.cc:458] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:20.224233  6037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:14:20.224905  6037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:14:20.225098   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:14:20.226305  6037 catalog_manager.cc:1347] Generated new cluster ID: b6ef669480b5437a9acdda5c69ed3be3
I20240426 13:14:20.226352  6037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:14:20.238996  6037 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:14:20.239434  6037 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:14:20.246743  6037 catalog_manager.cc:5857] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79: Generated new TSK 0
I20240426 13:14:20.246915  6037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:14:20.257107   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:20.259506  6052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:20.259956  6050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:20.260134  6049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:20.260983   398 server_base.cc:1034] running on GCE node
I20240426 13:14:20.261086   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:20.261116   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:20.261133   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137260261133 us; error 0 us; skew 500 ppm
I20240426 13:14:20.261857   398 webserver.cc:460] Webserver started at http://127.0.99.129:41527/ using document root <none> and password file <none>
I20240426 13:14:20.262001   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:20.262046   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:20.262177   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:20.262499   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "9bf678022cfb478ab68f85cb415d74d1"
format_stamp: "Formatted at 2024-04-26 13:14:20 on dist-test-slave-2jsl"
I20240426 13:14:20.263741   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:20.264564  6057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.264771   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:20.264837   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "9bf678022cfb478ab68f85cb415d74d1"
format_stamp: "Formatted at 2024-04-26 13:14:20 on dist-test-slave-2jsl"
I20240426 13:14:20.264894   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:20.277431   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:20.277701   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:20.277925   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:20.278304   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:20.278337   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.278373   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:20.278406   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.281901   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:45443
I20240426 13:14:20.281931  6120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:45443 every 8 connection(s)
I20240426 13:14:20.282528   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:20.284092  6125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:20.284508  6126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:20.285296  6128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:20.286099   398 server_base.cc:1034] running on GCE node
I20240426 13:14:20.286329   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:20.286389   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:20.286414   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137260286415 us; error 0 us; skew 500 ppm
I20240426 13:14:20.286458  6121 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34057
I20240426 13:14:20.286525  6121 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:20.286712  6121 heartbeater.cc:511] Master 127.0.99.190:34057 requested a full tablet report, sending...
I20240426 13:14:20.287295  5992 ts_manager.cc:194] Registered new tserver with Master: 9bf678022cfb478ab68f85cb415d74d1 (127.0.99.129:45443)
I20240426 13:14:20.287360   398 webserver.cc:460] Webserver started at http://127.0.99.130:43203/ using document root <none> and password file <none>
I20240426 13:14:20.287501   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:20.287571   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:20.287643   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:20.288043   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "dc8e84b8ae364944a308ed32ff23c0fb"
format_stamp: "Formatted at 2024-04-26 13:14:20 on dist-test-slave-2jsl"
I20240426 13:14:20.288122  5992 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36924
I20240426 13:14:20.289542   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:20.290323  6133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.290504   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.290581   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "dc8e84b8ae364944a308ed32ff23c0fb"
format_stamp: "Formatted at 2024-04-26 13:14:20 on dist-test-slave-2jsl"
I20240426 13:14:20.290645   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:20.302600   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:20.302850   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:20.303071   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:20.303443   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:20.303483   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.303532   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:20.303563   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.307611   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:39327
I20240426 13:14:20.307649  6196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:39327 every 8 connection(s)
I20240426 13:14:20.308311   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:20.309935  6200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:20.310999  6203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:20.311882   398 server_base.cc:1034] running on GCE node
I20240426 13:14:20.311954  6197 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34057
I20240426 13:14:20.312041  6197 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:20.312192  6197 heartbeater.cc:511] Master 127.0.99.190:34057 requested a full tablet report, sending...
W20240426 13:14:20.312371  6201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:20.312559   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:20.312606   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:20.312626   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137260312626 us; error 0 us; skew 500 ppm
I20240426 13:14:20.312721  5992 ts_manager.cc:194] Registered new tserver with Master: dc8e84b8ae364944a308ed32ff23c0fb (127.0.99.130:39327)
I20240426 13:14:20.313345  5992 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36928
I20240426 13:14:20.313452   398 webserver.cc:460] Webserver started at http://127.0.99.131:36945/ using document root <none> and password file <none>
I20240426 13:14:20.313591   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:20.313640   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:20.313704   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:20.314286   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "339b344e0f1d4e7cbcf973dc41b4ff70"
format_stamp: "Formatted at 2024-04-26 13:14:20 on dist-test-slave-2jsl"
I20240426 13:14:20.315744   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:20.316550  6208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.316746   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:14:20.316818   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "339b344e0f1d4e7cbcf973dc41b4ff70"
format_stamp: "Formatted at 2024-04-26 13:14:20 on dist-test-slave-2jsl"
I20240426 13:14:20.316902   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientCommitSequence.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:20.325696   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:20.325938   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:20.326191   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:20.326575   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:20.326614   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.326656   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:20.326692   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:20.330971   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:35885
I20240426 13:14:20.330991  6271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:35885 every 8 connection(s)
I20240426 13:14:20.334574  6272 heartbeater.cc:347] Connected to a master server at 127.0.99.190:34057
I20240426 13:14:20.334657  6272 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:20.334838  6272 heartbeater.cc:511] Master 127.0.99.190:34057 requested a full tablet report, sending...
I20240426 13:14:20.335322  5992 ts_manager.cc:194] Registered new tserver with Master: 339b344e0f1d4e7cbcf973dc41b4ff70 (127.0.99.131:35885)
I20240426 13:14:20.335750  5992 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36944
I20240426 13:14:20.335757   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004505504s
I20240426 13:14:20.336273   398 test_util.cc:268] Using random seed: -1367366512
I20240426 13:14:20.342803  5992 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36952:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:14:20.343148  5992 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:14:20.351217  6085 tablet_service.cc:1467] Processing CreateTablet for tablet 8e69bc30b49f4d4c87c0601e0650188b (DEFAULT_TABLE table=test-workload [id=2d3cce0d34e54d56bd527f9581ae865e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:20.351297  6236 tablet_service.cc:1467] Processing CreateTablet for tablet 8e69bc30b49f4d4c87c0601e0650188b (DEFAULT_TABLE table=test-workload [id=2d3cce0d34e54d56bd527f9581ae865e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:20.351457  6236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e69bc30b49f4d4c87c0601e0650188b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:20.351531  6161 tablet_service.cc:1467] Processing CreateTablet for tablet 8e69bc30b49f4d4c87c0601e0650188b (DEFAULT_TABLE table=test-workload [id=2d3cce0d34e54d56bd527f9581ae865e]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:20.351749  6161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e69bc30b49f4d4c87c0601e0650188b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:20.351457  6085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e69bc30b49f4d4c87c0601e0650188b. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:20.353401  6287 tablet_bootstrap.cc:492] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70: Bootstrap starting.
I20240426 13:14:20.353709  6288 tablet_bootstrap.cc:492] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb: Bootstrap starting.
I20240426 13:14:20.354418  6287 tablet_bootstrap.cc:654] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:20.354684  6288 tablet_bootstrap.cc:654] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:20.355387  6287 tablet_bootstrap.cc:492] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70: No bootstrap required, opened a new log
I20240426 13:14:20.355513  6287 ts_tablet_manager.cc:1367] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:14:20.355746  6291 tablet_bootstrap.cc:492] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1: Bootstrap starting.
I20240426 13:14:20.355823  6288 tablet_bootstrap.cc:492] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb: No bootstrap required, opened a new log
I20240426 13:14:20.355909  6288 ts_tablet_manager.cc:1367] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:14:20.356318  6288 raft_consensus.cc:348] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.356408  6287 raft_consensus.cc:348] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.356454  6288 raft_consensus.cc:374] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:20.356487  6287 raft_consensus.cc:374] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:20.356488  6288 raft_consensus.cc:729] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc8e84b8ae364944a308ed32ff23c0fb, State: Initialized, Role: FOLLOWER
I20240426 13:14:20.356540  6291 tablet_bootstrap.cc:654] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:20.356540  6287 raft_consensus.cc:729] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339b344e0f1d4e7cbcf973dc41b4ff70, State: Initialized, Role: FOLLOWER
I20240426 13:14:20.356668  6288 consensus_queue.cc:260] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.356707  6287 consensus_queue.cc:260] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.357095  6197 heartbeater.cc:503] Master 127.0.99.190:34057 was elected leader, sending a full tablet report...
I20240426 13:14:20.357168  6288 ts_tablet_manager.cc:1398] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:20.357206  6272 heartbeater.cc:503] Master 127.0.99.190:34057 was elected leader, sending a full tablet report...
I20240426 13:14:20.357084  6287 ts_tablet_manager.cc:1398] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:14:20.358284  6291 tablet_bootstrap.cc:492] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1: No bootstrap required, opened a new log
I20240426 13:14:20.358363  6291 ts_tablet_manager.cc:1367] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:14:20.358721  6291 raft_consensus.cc:348] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.358837  6291 raft_consensus.cc:374] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:20.358887  6291 raft_consensus.cc:729] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bf678022cfb478ab68f85cb415d74d1, State: Initialized, Role: FOLLOWER
I20240426 13:14:20.359004  6291 consensus_queue.cc:260] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.359326  6121 heartbeater.cc:503] Master 127.0.99.190:34057 was elected leader, sending a full tablet report...
I20240426 13:14:20.359328  6291 ts_tablet_manager.cc:1398] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:20.361994  6292 raft_consensus.cc:482] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:14:20.362080  6292 raft_consensus.cc:504] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.362430  6292 leader_election.cc:288] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bf678022cfb478ab68f85cb415d74d1 (127.0.99.129:45443), 339b344e0f1d4e7cbcf973dc41b4ff70 (127.0.99.131:35885)
I20240426 13:14:20.365249  6095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e69bc30b49f4d4c87c0601e0650188b" candidate_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bf678022cfb478ab68f85cb415d74d1" is_pre_election: true
I20240426 13:14:20.365365  6095 raft_consensus.cc:2454] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc8e84b8ae364944a308ed32ff23c0fb in term 0.
I20240426 13:14:20.365543  6246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e69bc30b49f4d4c87c0601e0650188b" candidate_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" is_pre_election: true
I20240426 13:14:20.365639  6134 leader_election.cc:302] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bf678022cfb478ab68f85cb415d74d1, dc8e84b8ae364944a308ed32ff23c0fb; no voters: 
I20240426 13:14:20.365644  6246 raft_consensus.cc:2454] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc8e84b8ae364944a308ed32ff23c0fb in term 0.
I20240426 13:14:20.365856  6292 raft_consensus.cc:2781] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:14:20.365932  6292 raft_consensus.cc:482] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:14:20.366010  6292 raft_consensus.cc:3037] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:20.366648  6292 raft_consensus.cc:504] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.366927  6292 leader_election.cc:288] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [CANDIDATE]: Term 1 election: Requested vote from peers 9bf678022cfb478ab68f85cb415d74d1 (127.0.99.129:45443), 339b344e0f1d4e7cbcf973dc41b4ff70 (127.0.99.131:35885)
I20240426 13:14:20.367177  6095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e69bc30b49f4d4c87c0601e0650188b" candidate_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bf678022cfb478ab68f85cb415d74d1"
I20240426 13:14:20.367218  6246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e69bc30b49f4d4c87c0601e0650188b" candidate_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70"
I20240426 13:14:20.367256  6095 raft_consensus.cc:3037] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:20.367332  6246 raft_consensus.cc:3037] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:20.368000  6095 raft_consensus.cc:2454] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc8e84b8ae364944a308ed32ff23c0fb in term 1.
I20240426 13:14:20.368256  6134 leader_election.cc:302] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9bf678022cfb478ab68f85cb415d74d1, dc8e84b8ae364944a308ed32ff23c0fb; no voters: 
I20240426 13:14:20.368278  6246 raft_consensus.cc:2454] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc8e84b8ae364944a308ed32ff23c0fb in term 1.
I20240426 13:14:20.368444  6292 raft_consensus.cc:2781] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:20.368634  6292 raft_consensus.cc:686] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 1 LEADER]: Becoming Leader. State: Replica: dc8e84b8ae364944a308ed32ff23c0fb, State: Running, Role: LEADER
I20240426 13:14:20.368789  6292 consensus_queue.cc:237] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:20.370245  5990 catalog_manager.cc:5484] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb reported cstate change: term changed from 0 to 1, leader changed from <none> to dc8e84b8ae364944a308ed32ff23c0fb (127.0.99.130). New cstate: current_term: 1 leader_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } health_report { overall_health: UNKNOWN } } }
I20240426 13:14:20.399085  6246 raft_consensus.cc:1261] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 1 FOLLOWER]: Refusing update from remote peer dc8e84b8ae364944a308ed32ff23c0fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:20.399149  6095 raft_consensus.cc:1261] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 1 FOLLOWER]: Refusing update from remote peer dc8e84b8ae364944a308ed32ff23c0fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:20.399420  6297 consensus_queue.cc:1035] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:20.399816  6297 consensus_queue.cc:1035] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:30.772817  6329 raft_consensus.cc:979] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb: : Instructing follower 9bf678022cfb478ab68f85cb415d74d1 to start an election
I20240426 13:14:30.772938  6329 raft_consensus.cc:1067] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 1 LEADER]: Signalling peer 9bf678022cfb478ab68f85cb415d74d1 to start an election
I20240426 13:14:30.773420  6095 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8e69bc30b49f4d4c87c0601e0650188b"
dest_uuid: "9bf678022cfb478ab68f85cb415d74d1"
 from {username='slave'} at 127.0.0.1:50062
I20240426 13:14:30.773537  6095 raft_consensus.cc:482] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:14:30.773591  6095 raft_consensus.cc:3037] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:14:30.774449  6095 raft_consensus.cc:504] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:30.774853  6095 leader_election.cc:288] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [CANDIDATE]: Term 2 election: Requested vote from peers dc8e84b8ae364944a308ed32ff23c0fb (127.0.99.130:39327), 339b344e0f1d4e7cbcf973dc41b4ff70 (127.0.99.131:35885)
I20240426 13:14:30.778139  6171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e69bc30b49f4d4c87c0601e0650188b" candidate_uuid: "9bf678022cfb478ab68f85cb415d74d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3029 } } ignore_live_leader: true dest_uuid: "dc8e84b8ae364944a308ed32ff23c0fb"
I20240426 13:14:30.778255  6171 raft_consensus.cc:3032] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:14:30.778311  6171 raft_consensus.cc:729] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: dc8e84b8ae364944a308ed32ff23c0fb, State: Running, Role: LEADER
I20240426 13:14:30.778291  6245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e69bc30b49f4d4c87c0601e0650188b" candidate_uuid: "9bf678022cfb478ab68f85cb415d74d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3029 } } ignore_live_leader: true dest_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70"
I20240426 13:14:30.778415  6245 raft_consensus.cc:3037] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:14:30.778483  6171 consensus_queue.cc:260] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3029, Committed index: 3029, Last appended: 1.3029, Last appended by leader: 3029, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:30.778641  6171 raft_consensus.cc:3037] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:14:30.779223  6245 raft_consensus.cc:2454] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bf678022cfb478ab68f85cb415d74d1 in term 2.
I20240426 13:14:30.779386  6171 raft_consensus.cc:2454] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bf678022cfb478ab68f85cb415d74d1 in term 2.
I20240426 13:14:30.779533  6060 leader_election.cc:302] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 339b344e0f1d4e7cbcf973dc41b4ff70, 9bf678022cfb478ab68f85cb415d74d1; no voters: 
I20240426 13:14:30.779856  6350 raft_consensus.cc:2781] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:14:30.780020  6350 raft_consensus.cc:686] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 2 LEADER]: Becoming Leader. State: Replica: 9bf678022cfb478ab68f85cb415d74d1, State: Running, Role: LEADER
I20240426 13:14:30.780150  6350 consensus_queue.cc:237] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3029, Committed index: 3029, Last appended: 1.3029, Last appended by leader: 3029, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } }
I20240426 13:14:30.781390  5991 catalog_manager.cc:5484] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 reported cstate change: term changed from 1 to 2, leader changed from dc8e84b8ae364944a308ed32ff23c0fb (127.0.99.130) to 9bf678022cfb478ab68f85cb415d74d1 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "9bf678022cfb478ab68f85cb415d74d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9bf678022cfb478ab68f85cb415d74d1" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 45443 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 } health_report { overall_health: UNKNOWN } } }
I20240426 13:14:30.796129  6245 raft_consensus.cc:1261] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 2 FOLLOWER]: Refusing update from remote peer 9bf678022cfb478ab68f85cb415d74d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3029. Preceding OpId from leader: term: 2 index: 3031. (index mismatch)
I20240426 13:14:30.796129  6171 raft_consensus.cc:1261] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 2 FOLLOWER]: Refusing update from remote peer 9bf678022cfb478ab68f85cb415d74d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3029. Preceding OpId from leader: term: 2 index: 3031. (index mismatch)
I20240426 13:14:30.796543  6350 consensus_queue.cc:1035] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc8e84b8ae364944a308ed32ff23c0fb" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 39327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3030, Last known committed idx: 3029, Time since last communication: 0.000s
I20240426 13:14:30.796680  6351 consensus_queue.cc:1035] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "339b344e0f1d4e7cbcf973dc41b4ff70" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3030, Last known committed idx: 3029, Time since last communication: 0.000s
W20240426 13:14:30.800837  6307 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54686: Illegal state: Cannot finalize transaction in state: OPEN
W20240426 13:14:30.801733  6307 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54686: Illegal state: Transaction 0 already open
W20240426 13:14:30.804973  6307 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54686: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:14:30.805838  6307 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54686: Illegal state: Transaction 0 commit already in progress
W20240426 13:14:30.809655  6307 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54686: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:14:30.810460  6307 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54686: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:14:30.811344  6307 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54686: Illegal state: Transaction 0 commit already in progress
W20240426 13:14:30.812088  6307 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:54686: Illegal state: Transaction metadata for transaction 0 already set
I20240426 13:14:30.812687   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:14:30.816426   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:30.816596   398 tablet_replica.cc:330] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1: stopping tablet replica
I20240426 13:14:30.816697   398 raft_consensus.cc:2229] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:14:30.816817   398 raft_consensus.cc:2258] T 8e69bc30b49f4d4c87c0601e0650188b P 9bf678022cfb478ab68f85cb415d74d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:30.832734   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:14:30.834213   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:14:30.837612   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:30.837743   398 tablet_replica.cc:330] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb: stopping tablet replica
I20240426 13:14:30.837814   398 raft_consensus.cc:2229] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:30.837883   398 raft_consensus.cc:2258] T 8e69bc30b49f4d4c87c0601e0650188b P dc8e84b8ae364944a308ed32ff23c0fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:30.853336   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:14:30.854871   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:14:30.858747   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:30.858862   398 tablet_replica.cc:330] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70: stopping tablet replica
I20240426 13:14:30.858942   398 raft_consensus.cc:2229] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:30.859015   398 raft_consensus.cc:2258] T 8e69bc30b49f4d4c87c0601e0650188b P 339b344e0f1d4e7cbcf973dc41b4ff70 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:30.875648   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:14:30.877092   398 master.cc:537] Master@127.0.99.190:34057 shutting down...
I20240426 13:14:30.880066   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:30.880170   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:30.880231   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P b02e412889e1467eb0e0984f36668c79: stopping tablet replica
I20240426 13:14:30.892458   398 master.cc:552] Master@127.0.99.190:34057 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientCommitSequence (10701 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientAbortSequence
I20240426 13:14:30.900493   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:44681
I20240426 13:14:30.900760   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:30.902730  6368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:30.903036  6365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:30.903504  6366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:30.904417   398 server_base.cc:1034] running on GCE node
I20240426 13:14:30.904537   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:30.904567   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:30.904580   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137270904581 us; error 0 us; skew 500 ppm
I20240426 13:14:30.905357   398 webserver.cc:460] Webserver started at http://127.0.99.190:33111/ using document root <none> and password file <none>
I20240426 13:14:30.905479   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:30.905524   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:30.905576   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:30.905941   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "2338df71df0648fda7bb67c5811b6971"
format_stamp: "Formatted at 2024-04-26 13:14:30 on dist-test-slave-2jsl"
I20240426 13:14:30.907279   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:30.908098  6373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:30.908308   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:30.908371   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "2338df71df0648fda7bb67c5811b6971"
format_stamp: "Formatted at 2024-04-26 13:14:30 on dist-test-slave-2jsl"
I20240426 13:14:30.908437   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:30.917021   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:30.917307   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:30.921202   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:44681
I20240426 13:14:30.927704  6425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:44681 every 8 connection(s)
I20240426 13:14:30.928056  6426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:30.929457  6426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971: Bootstrap starting.
I20240426 13:14:30.930081  6426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:30.930817  6426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971: No bootstrap required, opened a new log
I20240426 13:14:30.931102  6426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338df71df0648fda7bb67c5811b6971" member_type: VOTER }
I20240426 13:14:30.931167  6426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:30.931205  6426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2338df71df0648fda7bb67c5811b6971, State: Initialized, Role: FOLLOWER
I20240426 13:14:30.931310  6426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338df71df0648fda7bb67c5811b6971" member_type: VOTER }
I20240426 13:14:30.931366  6426 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:14:30.931386  6426 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:14:30.931412  6426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:30.931996  6426 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338df71df0648fda7bb67c5811b6971" member_type: VOTER }
I20240426 13:14:30.932126  6426 leader_election.cc:302] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2338df71df0648fda7bb67c5811b6971; no voters: 
I20240426 13:14:30.932303  6426 leader_election.cc:288] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:14:30.932432  6429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:30.932650  6429 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 1 LEADER]: Becoming Leader. State: Replica: 2338df71df0648fda7bb67c5811b6971, State: Running, Role: LEADER
I20240426 13:14:30.932760  6426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:14:30.932821  6429 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338df71df0648fda7bb67c5811b6971" member_type: VOTER }
I20240426 13:14:30.933331  6429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2338df71df0648fda7bb67c5811b6971. Latest consensus state: current_term: 1 leader_uuid: "2338df71df0648fda7bb67c5811b6971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338df71df0648fda7bb67c5811b6971" member_type: VOTER } }
I20240426 13:14:30.933568  6429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:30.933491  6431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2338df71df0648fda7bb67c5811b6971" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338df71df0648fda7bb67c5811b6971" member_type: VOTER } }
I20240426 13:14:30.933825  6431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:30.934453   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20240426 13:14:30.935009  6445 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20240426 13:14:30.935077  6445 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20240426 13:14:30.935140  6438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:14:30.935601  6438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:14:30.937126  6438 catalog_manager.cc:1347] Generated new cluster ID: ac514e2ae42c49f1825bc906f8d6b8e0
I20240426 13:14:30.937181  6438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:14:30.951583  6438 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:14:30.951989  6438 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:14:30.959517  6438 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971: Generated new TSK 0
I20240426 13:14:30.959637  6438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:14:30.966421   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:30.968297  6448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:30.969239  6450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:30.969431  6447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:30.970306   398 server_base.cc:1034] running on GCE node
I20240426 13:14:30.970424   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:30.970455   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:30.970474   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137270970473 us; error 0 us; skew 500 ppm
I20240426 13:14:30.971285   398 webserver.cc:460] Webserver started at http://127.0.99.129:33433/ using document root <none> and password file <none>
I20240426 13:14:30.971421   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:30.971468   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:30.971539   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:30.971899   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "f76af4da4bb04ff9a295d9bd08867d78"
format_stamp: "Formatted at 2024-04-26 13:14:30 on dist-test-slave-2jsl"
I20240426 13:14:30.973127   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:30.973873  6455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:30.974103   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:30.974174   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "f76af4da4bb04ff9a295d9bd08867d78"
format_stamp: "Formatted at 2024-04-26 13:14:30 on dist-test-slave-2jsl"
I20240426 13:14:30.974234   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:30.983011   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:30.983253   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:30.983458   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:30.983799   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:30.983843   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:30.983881   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:30.983911   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:30.987274   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:38311
I20240426 13:14:30.987293  6518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:38311 every 8 connection(s)
I20240426 13:14:30.987998   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:30.989991  6523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:30.990466  6524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:30.991376  6526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:30.992169   398 server_base.cc:1034] running on GCE node
I20240426 13:14:30.992322  6519 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44681
I20240426 13:14:30.992339   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:30.992405   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:30.992426   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137270992426 us; error 0 us; skew 500 ppm
I20240426 13:14:30.992439  6519 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:30.992647  6519 heartbeater.cc:511] Master 127.0.99.190:44681 requested a full tablet report, sending...
I20240426 13:14:30.993265  6390 ts_manager.cc:194] Registered new tserver with Master: f76af4da4bb04ff9a295d9bd08867d78 (127.0.99.129:38311)
I20240426 13:14:30.993597   398 webserver.cc:460] Webserver started at http://127.0.99.130:42899/ using document root <none> and password file <none>
I20240426 13:14:30.993750   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:30.993803   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:30.993878   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:30.994160  6390 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50062
I20240426 13:14:30.994244   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "ca73f37c45a24eecad3903081da19d76"
format_stamp: "Formatted at 2024-04-26 13:14:30 on dist-test-slave-2jsl"
I20240426 13:14:30.995568   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:30.996397  6531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:30.996694   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:30.996759   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "ca73f37c45a24eecad3903081da19d76"
format_stamp: "Formatted at 2024-04-26 13:14:30 on dist-test-slave-2jsl"
I20240426 13:14:30.996824   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:31.011494   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:31.011781   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:31.012009   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:31.012372   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:31.012408   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:31.012452   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:31.012483   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:31.016229   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:40137
I20240426 13:14:31.016265  6594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:40137 every 8 connection(s)
I20240426 13:14:31.016964   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:31.019207  6599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:31.019340  6598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:31.020226  6601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:31.020779  6595 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44681
I20240426 13:14:31.020867  6595 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:31.021055   398 server_base.cc:1034] running on GCE node
I20240426 13:14:31.021142  6595 heartbeater.cc:511] Master 127.0.99.190:44681 requested a full tablet report, sending...
I20240426 13:14:31.021181   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:31.021225   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:31.021253   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137271021253 us; error 0 us; skew 500 ppm
I20240426 13:14:31.021741  6390 ts_manager.cc:194] Registered new tserver with Master: ca73f37c45a24eecad3903081da19d76 (127.0.99.130:40137)
I20240426 13:14:31.022167   398 webserver.cc:460] Webserver started at http://127.0.99.131:38705/ using document root <none> and password file <none>
I20240426 13:14:31.022300   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:31.022342   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:31.022403   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:31.022429  6390 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50078
I20240426 13:14:31.022784   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "619c515bc7494795b9ddf93c63713988"
format_stamp: "Formatted at 2024-04-26 13:14:31 on dist-test-slave-2jsl"
I20240426 13:14:31.024137   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:31.024966  6606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:31.025220   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:31.025285   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "619c515bc7494795b9ddf93c63713988"
format_stamp: "Formatted at 2024-04-26 13:14:31 on dist-test-slave-2jsl"
I20240426 13:14:31.025362   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientAbortSequence.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:31.048663   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:31.048940   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:31.049180   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:31.049580   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:31.049616   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:31.049669   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:31.049702   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:31.053812   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:32971
I20240426 13:14:31.053864  6669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:32971 every 8 connection(s)
I20240426 13:14:31.057832  6670 heartbeater.cc:347] Connected to a master server at 127.0.99.190:44681
I20240426 13:14:31.057915  6670 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:31.058117  6670 heartbeater.cc:511] Master 127.0.99.190:44681 requested a full tablet report, sending...
I20240426 13:14:31.058672  6390 ts_manager.cc:194] Registered new tserver with Master: 619c515bc7494795b9ddf93c63713988 (127.0.99.131:32971)
I20240426 13:14:31.058717   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00453112s
I20240426 13:14:31.059222   398 test_util.cc:268] Using random seed: -1356643562
I20240426 13:14:31.059320  6390 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50080
I20240426 13:14:31.065719  6390 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50096:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:14:31.066082  6390 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:14:31.074427  6634 tablet_service.cc:1467] Processing CreateTablet for tablet b529e2d334f24b5781489452026bc068 (DEFAULT_TABLE table=test-workload [id=f1edad28429048aca5ec6de53ce57a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:31.074539  6483 tablet_service.cc:1467] Processing CreateTablet for tablet b529e2d334f24b5781489452026bc068 (DEFAULT_TABLE table=test-workload [id=f1edad28429048aca5ec6de53ce57a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:31.074702  6634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b529e2d334f24b5781489452026bc068. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:31.074805  6483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b529e2d334f24b5781489452026bc068. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:31.076586  6559 tablet_service.cc:1467] Processing CreateTablet for tablet b529e2d334f24b5781489452026bc068 (DEFAULT_TABLE table=test-workload [id=f1edad28429048aca5ec6de53ce57a76]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:31.076823  6559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b529e2d334f24b5781489452026bc068. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:31.077113  6686 tablet_bootstrap.cc:492] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988: Bootstrap starting.
I20240426 13:14:31.077366  6685 tablet_bootstrap.cc:492] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78: Bootstrap starting.
I20240426 13:14:31.077909  6686 tablet_bootstrap.cc:654] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:31.078320  6685 tablet_bootstrap.cc:654] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:31.078832  6688 tablet_bootstrap.cc:492] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76: Bootstrap starting.
I20240426 13:14:31.079562  6686 tablet_bootstrap.cc:492] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988: No bootstrap required, opened a new log
I20240426 13:14:31.079635  6686 ts_tablet_manager.cc:1367] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:14:31.079672  6688 tablet_bootstrap.cc:654] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:31.079669  6685 tablet_bootstrap.cc:492] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78: No bootstrap required, opened a new log
I20240426 13:14:31.079815  6685 ts_tablet_manager.cc:1367] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:14:31.080020  6686 raft_consensus.cc:348] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.080103  6686 raft_consensus.cc:374] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:31.080142  6686 raft_consensus.cc:729] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 619c515bc7494795b9ddf93c63713988, State: Initialized, Role: FOLLOWER
I20240426 13:14:31.080222  6685 raft_consensus.cc:348] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.080250  6686 consensus_queue.cc:260] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.080329  6685 raft_consensus.cc:374] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:31.080369  6685 raft_consensus.cc:729] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76af4da4bb04ff9a295d9bd08867d78, State: Initialized, Role: FOLLOWER
I20240426 13:14:31.080480  6685 consensus_queue.cc:260] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.080596  6686 ts_tablet_manager.cc:1398] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:31.080744  6670 heartbeater.cc:503] Master 127.0.99.190:44681 was elected leader, sending a full tablet report...
I20240426 13:14:31.080770  6685 ts_tablet_manager.cc:1398] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:31.080791  6688 tablet_bootstrap.cc:492] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76: No bootstrap required, opened a new log
I20240426 13:14:31.080852  6688 ts_tablet_manager.cc:1367] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:14:31.080897  6519 heartbeater.cc:503] Master 127.0.99.190:44681 was elected leader, sending a full tablet report...
I20240426 13:14:31.081210  6688 raft_consensus.cc:348] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.081306  6688 raft_consensus.cc:374] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:31.081347  6688 raft_consensus.cc:729] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca73f37c45a24eecad3903081da19d76, State: Initialized, Role: FOLLOWER
I20240426 13:14:31.081471  6688 consensus_queue.cc:260] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.081790  6688 ts_tablet_manager.cc:1398] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:31.081923  6595 heartbeater.cc:503] Master 127.0.99.190:44681 was elected leader, sending a full tablet report...
I20240426 13:14:31.115090  6692 raft_consensus.cc:482] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:14:31.115175  6692 raft_consensus.cc:504] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.115585  6692 leader_election.cc:288] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 619c515bc7494795b9ddf93c63713988 (127.0.99.131:32971), ca73f37c45a24eecad3903081da19d76 (127.0.99.130:40137)
I20240426 13:14:31.118479  6644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b529e2d334f24b5781489452026bc068" candidate_uuid: "f76af4da4bb04ff9a295d9bd08867d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619c515bc7494795b9ddf93c63713988" is_pre_election: true
I20240426 13:14:31.118599  6644 raft_consensus.cc:2454] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76af4da4bb04ff9a295d9bd08867d78 in term 0.
I20240426 13:14:31.118641  6569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b529e2d334f24b5781489452026bc068" candidate_uuid: "f76af4da4bb04ff9a295d9bd08867d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca73f37c45a24eecad3903081da19d76" is_pre_election: true
I20240426 13:14:31.118752  6569 raft_consensus.cc:2454] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76af4da4bb04ff9a295d9bd08867d78 in term 0.
I20240426 13:14:31.118903  6456 leader_election.cc:302] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 619c515bc7494795b9ddf93c63713988, f76af4da4bb04ff9a295d9bd08867d78; no voters: 
I20240426 13:14:31.119071  6692 raft_consensus.cc:2781] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:14:31.119132  6692 raft_consensus.cc:482] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:14:31.119160  6692 raft_consensus.cc:3037] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:31.119829  6692 raft_consensus.cc:504] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.120110  6692 leader_election.cc:288] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [CANDIDATE]: Term 1 election: Requested vote from peers 619c515bc7494795b9ddf93c63713988 (127.0.99.131:32971), ca73f37c45a24eecad3903081da19d76 (127.0.99.130:40137)
I20240426 13:14:31.120344  6644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b529e2d334f24b5781489452026bc068" candidate_uuid: "f76af4da4bb04ff9a295d9bd08867d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "619c515bc7494795b9ddf93c63713988"
I20240426 13:14:31.120369  6569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b529e2d334f24b5781489452026bc068" candidate_uuid: "f76af4da4bb04ff9a295d9bd08867d78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca73f37c45a24eecad3903081da19d76"
I20240426 13:14:31.120432  6644 raft_consensus.cc:3037] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:31.120445  6569 raft_consensus.cc:3037] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:31.121248  6569 raft_consensus.cc:2454] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76af4da4bb04ff9a295d9bd08867d78 in term 1.
I20240426 13:14:31.121260  6644 raft_consensus.cc:2454] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76af4da4bb04ff9a295d9bd08867d78 in term 1.
I20240426 13:14:31.121521  6458 leader_election.cc:302] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ca73f37c45a24eecad3903081da19d76, f76af4da4bb04ff9a295d9bd08867d78; no voters: 
I20240426 13:14:31.121729  6692 raft_consensus.cc:2781] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:31.121878  6692 raft_consensus.cc:686] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 1 LEADER]: Becoming Leader. State: Replica: f76af4da4bb04ff9a295d9bd08867d78, State: Running, Role: LEADER
I20240426 13:14:31.122013  6692 consensus_queue.cc:237] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } }
I20240426 13:14:31.123252  6389 catalog_manager.cc:5484] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 reported cstate change: term changed from 0 to 1, leader changed from <none> to f76af4da4bb04ff9a295d9bd08867d78 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "f76af4da4bb04ff9a295d9bd08867d78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76af4da4bb04ff9a295d9bd08867d78" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38311 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 } health_report { overall_health: UNKNOWN } } }
I20240426 13:14:31.149741  6569 raft_consensus.cc:1261] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 1 FOLLOWER]: Refusing update from remote peer f76af4da4bb04ff9a295d9bd08867d78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:31.150046  6644 raft_consensus.cc:1261] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 1 FOLLOWER]: Refusing update from remote peer f76af4da4bb04ff9a295d9bd08867d78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:31.150105  6695 consensus_queue.cc:1035] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca73f37c45a24eecad3903081da19d76" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:31.150358  6692 consensus_queue.cc:1035] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "619c515bc7494795b9ddf93c63713988" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 32971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240426 13:14:41.180028  6702 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60222: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:14:41.180800  6702 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60222: Illegal state: Transaction metadata for transaction 1 already exists
W20240426 13:14:41.181525  6702 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60222: Illegal state: Cannot begin committing transaction in state: INITIALIZING
W20240426 13:14:41.182271  6702 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60222: Illegal state: Cannot begin committing transaction in state: INITIALIZING
W20240426 13:14:41.183005  6702 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60222: Illegal state: Transaction metadata for transaction 0 already exists
W20240426 13:14:41.183804  6702 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60222: Illegal state: Transaction metadata for transaction 1 already exists
W20240426 13:14:41.184641  6702 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60222: Illegal state: Transaction metadata for transaction 0 already set
W20240426 13:14:41.185396  6702 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60222: Illegal state: Transaction metadata for transaction 1 already set
I20240426 13:14:41.186029   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:14:41.190061   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:41.190212   398 tablet_replica.cc:330] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78: stopping tablet replica
I20240426 13:14:41.190284   398 raft_consensus.cc:2229] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:41.190388   398 raft_consensus.cc:2258] T b529e2d334f24b5781489452026bc068 P f76af4da4bb04ff9a295d9bd08867d78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:41.195991   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:14:41.197515   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:14:41.200820   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:41.200943   398 tablet_replica.cc:330] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76: stopping tablet replica
I20240426 13:14:41.201018   398 raft_consensus.cc:2229] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:41.201138   398 raft_consensus.cc:2258] T b529e2d334f24b5781489452026bc068 P ca73f37c45a24eecad3903081da19d76 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:41.205865   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:14:41.207075   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:14:41.210510   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:41.210634   398 tablet_replica.cc:330] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988: stopping tablet replica
I20240426 13:14:41.210716   398 raft_consensus.cc:2229] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:41.210799   398 raft_consensus.cc:2258] T b529e2d334f24b5781489452026bc068 P 619c515bc7494795b9ddf93c63713988 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:41.226764   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:14:41.228024   398 master.cc:537] Master@127.0.99.190:44681 shutting down...
I20240426 13:14:41.230701   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:41.230789   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:41.230829   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2338df71df0648fda7bb67c5811b6971: stopping tablet replica
I20240426 13:14:41.242460   398 master.cc:552] Master@127.0.99.190:44681 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientAbortSequence (10349 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun
I20240426 13:14:41.249951   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:40671
I20240426 13:14:41.250252   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:41.252012  6753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:41.252929  6754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:41.253523  6756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:41.254289   398 server_base.cc:1034] running on GCE node
I20240426 13:14:41.254415   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:41.254444   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:41.254464   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137281254463 us; error 0 us; skew 500 ppm
I20240426 13:14:41.255165   398 webserver.cc:460] Webserver started at http://127.0.99.190:37659/ using document root <none> and password file <none>
I20240426 13:14:41.255281   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:41.255321   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:41.255385   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:41.255739   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "0521690ffdd34ab0ad70295db544bbc5"
format_stamp: "Formatted at 2024-04-26 13:14:41 on dist-test-slave-2jsl"
I20240426 13:14:41.256942   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.001s
I20240426 13:14:41.257719  6761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.257920   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:41.257995   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "0521690ffdd34ab0ad70295db544bbc5"
format_stamp: "Formatted at 2024-04-26 13:14:41 on dist-test-slave-2jsl"
I20240426 13:14:41.258055   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:41.284929   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:41.285193   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:41.288446   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:40671
I20240426 13:14:41.293928  6814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:41.293951  6813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:40671 every 8 connection(s)
I20240426 13:14:41.295269  6814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5: Bootstrap starting.
I20240426 13:14:41.295842  6814 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:41.296578  6814 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5: No bootstrap required, opened a new log
I20240426 13:14:41.296912  6814 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0521690ffdd34ab0ad70295db544bbc5" member_type: VOTER }
I20240426 13:14:41.296999  6814 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:41.297029  6814 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0521690ffdd34ab0ad70295db544bbc5, State: Initialized, Role: FOLLOWER
I20240426 13:14:41.297171  6814 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0521690ffdd34ab0ad70295db544bbc5" member_type: VOTER }
I20240426 13:14:41.297235  6814 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:14:41.297277  6814 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:14:41.297324  6814 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:41.297885  6814 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0521690ffdd34ab0ad70295db544bbc5" member_type: VOTER }
I20240426 13:14:41.298013  6814 leader_election.cc:302] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0521690ffdd34ab0ad70295db544bbc5; no voters: 
I20240426 13:14:41.298175  6814 leader_election.cc:288] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:14:41.298271  6817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:41.298488  6817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 1 LEADER]: Becoming Leader. State: Replica: 0521690ffdd34ab0ad70295db544bbc5, State: Running, Role: LEADER
I20240426 13:14:41.298543  6814 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:14:41.298604  6817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0521690ffdd34ab0ad70295db544bbc5" member_type: VOTER }
I20240426 13:14:41.298998  6818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0521690ffdd34ab0ad70295db544bbc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0521690ffdd34ab0ad70295db544bbc5" member_type: VOTER } }
I20240426 13:14:41.299041  6819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0521690ffdd34ab0ad70295db544bbc5. Latest consensus state: current_term: 1 leader_uuid: "0521690ffdd34ab0ad70295db544bbc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0521690ffdd34ab0ad70295db544bbc5" member_type: VOTER } }
I20240426 13:14:41.299176  6818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:41.299197  6819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:41.299643  6826 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:14:41.300341  6826 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:14:41.300542   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:14:41.301656  6826 catalog_manager.cc:1347] Generated new cluster ID: 8c9b6fa1a0c84eb288dd9749af8ecfe2
I20240426 13:14:41.301723  6826 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:14:41.313642  6826 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:14:41.314145  6826 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:14:41.318163  6826 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5: Generated new TSK 0
I20240426 13:14:41.318284  6826 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:14:41.332665   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20240426 13:14:41.335053   398 server_base.cc:1034] running on GCE node
W20240426 13:14:41.335748  6838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:41.336203  6836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:41.336234  6835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:41.336448   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:41.336503   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:41.336525   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137281336524 us; error 0 us; skew 500 ppm
I20240426 13:14:41.337363   398 webserver.cc:460] Webserver started at http://127.0.99.129:37481/ using document root <none> and password file <none>
I20240426 13:14:41.337507   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:41.337561   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:41.337635   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:41.337997   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "5ea4615254e241e1b041b47a1d27da61"
format_stamp: "Formatted at 2024-04-26 13:14:41 on dist-test-slave-2jsl"
I20240426 13:14:41.339327   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:41.340164  6843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.340363   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:41.340423   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "5ea4615254e241e1b041b47a1d27da61"
format_stamp: "Formatted at 2024-04-26 13:14:41 on dist-test-slave-2jsl"
I20240426 13:14:41.340502   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:41.346199   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:41.346506   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:41.346741   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:41.347121   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:41.347160   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.347203   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:41.347237   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.351190   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:38777
I20240426 13:14:41.351207  6906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:38777 every 8 connection(s)
I20240426 13:14:41.351889   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:41.353415  6911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:41.354064  6912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:41.354843  6914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:41.355623   398 server_base.cc:1034] running on GCE node
I20240426 13:14:41.355770   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:41.355810   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:41.355829   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137281355829 us; error 0 us; skew 500 ppm
I20240426 13:14:41.355927  6907 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40671
I20240426 13:14:41.356001  6907 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:41.356175  6907 heartbeater.cc:511] Master 127.0.99.190:40671 requested a full tablet report, sending...
I20240426 13:14:41.356665   398 webserver.cc:460] Webserver started at http://127.0.99.130:41071/ using document root <none> and password file <none>
I20240426 13:14:41.356747  6778 ts_manager.cc:194] Registered new tserver with Master: 5ea4615254e241e1b041b47a1d27da61 (127.0.99.129:38777)
I20240426 13:14:41.356882   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:41.356948   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:41.357021   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:41.357424   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "1738567a9fd4445585dc1c48431f6fab"
format_stamp: "Formatted at 2024-04-26 13:14:41 on dist-test-slave-2jsl"
I20240426 13:14:41.357450  6778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46142
I20240426 13:14:41.358808   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:14:41.359680  6919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.359882   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:41.359946   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "1738567a9fd4445585dc1c48431f6fab"
format_stamp: "Formatted at 2024-04-26 13:14:41 on dist-test-slave-2jsl"
I20240426 13:14:41.360011   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:41.368707   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:41.368988   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:41.369215   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:41.369572   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:41.369608   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.369653   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:41.369685   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.373322   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:40217
I20240426 13:14:41.373355  6982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:40217 every 8 connection(s)
I20240426 13:14:41.374022   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:41.375802  6987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:41.376420  6986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:41.377291  6989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:41.377547  6983 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40671
I20240426 13:14:41.377615  6983 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:41.377821  6983 heartbeater.cc:511] Master 127.0.99.190:40671 requested a full tablet report, sending...
I20240426 13:14:41.378125   398 server_base.cc:1034] running on GCE node
I20240426 13:14:41.378291   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:41.378326   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:41.378340   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137281378340 us; error 0 us; skew 500 ppm
I20240426 13:14:41.378379  6778 ts_manager.cc:194] Registered new tserver with Master: 1738567a9fd4445585dc1c48431f6fab (127.0.99.130:40217)
I20240426 13:14:41.379068  6778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46152
I20240426 13:14:41.379184   398 webserver.cc:460] Webserver started at http://127.0.99.131:34031/ using document root <none> and password file <none>
I20240426 13:14:41.379308   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:41.379354   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:41.379426   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:41.379808   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "2e16f9fbc821433daee440eafa105a97"
format_stamp: "Formatted at 2024-04-26 13:14:41 on dist-test-slave-2jsl"
I20240426 13:14:41.380996   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:41.381681  6994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.381929   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:41.382017   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "2e16f9fbc821433daee440eafa105a97"
format_stamp: "Formatted at 2024-04-26 13:14:41 on dist-test-slave-2jsl"
I20240426 13:14:41.382077   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:41.389873   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:41.390249   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:41.390483   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:41.390868   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:41.390906   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.390942   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:41.390975   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:41.395378   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:38365
I20240426 13:14:41.395397  7057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:38365 every 8 connection(s)
I20240426 13:14:41.399012  7058 heartbeater.cc:347] Connected to a master server at 127.0.99.190:40671
I20240426 13:14:41.399118  7058 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:41.399427  7058 heartbeater.cc:511] Master 127.0.99.190:40671 requested a full tablet report, sending...
I20240426 13:14:41.399978  6778 ts_manager.cc:194] Registered new tserver with Master: 2e16f9fbc821433daee440eafa105a97 (127.0.99.131:38365)
I20240426 13:14:41.400204   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004475024s
I20240426 13:14:41.400568  6778 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46160
I20240426 13:14:41.400701   398 test_util.cc:268] Using random seed: -1346302084
I20240426 13:14:41.406922  6778 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46174:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:14:41.407281  6778 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:14:41.415642  6871 tablet_service.cc:1467] Processing CreateTablet for tablet 1738137e13f24b1a9a1e50ab0b0f679f (DEFAULT_TABLE table=test-workload [id=0600effcd60e4f37a04c4383f341c64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:41.415907  6871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1738137e13f24b1a9a1e50ab0b0f679f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:41.415944  6947 tablet_service.cc:1467] Processing CreateTablet for tablet 1738137e13f24b1a9a1e50ab0b0f679f (DEFAULT_TABLE table=test-workload [id=0600effcd60e4f37a04c4383f341c64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:41.416071  7022 tablet_service.cc:1467] Processing CreateTablet for tablet 1738137e13f24b1a9a1e50ab0b0f679f (DEFAULT_TABLE table=test-workload [id=0600effcd60e4f37a04c4383f341c64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:41.416182  6947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1738137e13f24b1a9a1e50ab0b0f679f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:41.416288  7022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1738137e13f24b1a9a1e50ab0b0f679f. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:41.417891  7073 tablet_bootstrap.cc:492] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab: Bootstrap starting.
I20240426 13:14:41.418161  7074 tablet_bootstrap.cc:492] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61: Bootstrap starting.
I20240426 13:14:41.419106  7073 tablet_bootstrap.cc:654] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:41.419106  7074 tablet_bootstrap.cc:654] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:41.419241  7075 tablet_bootstrap.cc:492] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97: Bootstrap starting.
I20240426 13:14:41.420068  7075 tablet_bootstrap.cc:654] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:41.420164  7073 tablet_bootstrap.cc:492] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab: No bootstrap required, opened a new log
I20240426 13:14:41.420258  7073 ts_tablet_manager.cc:1367] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:14:41.420634  7073 raft_consensus.cc:348] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.420742  7073 raft_consensus.cc:374] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:41.420786  7073 raft_consensus.cc:729] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1738567a9fd4445585dc1c48431f6fab, State: Initialized, Role: FOLLOWER
I20240426 13:14:41.420974  7075 tablet_bootstrap.cc:492] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97: No bootstrap required, opened a new log
I20240426 13:14:41.420928  7073 consensus_queue.cc:260] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.421054  7075 ts_tablet_manager.cc:1367] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:14:41.421258  7073 ts_tablet_manager.cc:1398] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:41.421365  6983 heartbeater.cc:503] Master 127.0.99.190:40671 was elected leader, sending a full tablet report...
I20240426 13:14:41.421403  7075 raft_consensus.cc:348] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.421485  7075 raft_consensus.cc:374] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:41.421519  7075 raft_consensus.cc:729] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e16f9fbc821433daee440eafa105a97, State: Initialized, Role: FOLLOWER
I20240426 13:14:41.421623  7075 consensus_queue.cc:260] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.421943  7075 ts_tablet_manager.cc:1398] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:41.422127  7058 heartbeater.cc:503] Master 127.0.99.190:40671 was elected leader, sending a full tablet report...
I20240426 13:14:41.422845  7074 tablet_bootstrap.cc:492] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61: No bootstrap required, opened a new log
I20240426 13:14:41.422909  7074 ts_tablet_manager.cc:1367] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61: Time spent bootstrapping tablet: real 0.005s	user 0.001s	sys 0.000s
I20240426 13:14:41.423198  7074 raft_consensus.cc:348] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.423280  7074 raft_consensus.cc:374] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:41.423310  7074 raft_consensus.cc:729] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ea4615254e241e1b041b47a1d27da61, State: Initialized, Role: FOLLOWER
I20240426 13:14:41.423413  7074 consensus_queue.cc:260] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.423766  7074 ts_tablet_manager.cc:1398] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:41.423774  6907 heartbeater.cc:503] Master 127.0.99.190:40671 was elected leader, sending a full tablet report...
I20240426 13:14:41.472892  7081 raft_consensus.cc:482] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:14:41.472965  7081 raft_consensus.cc:504] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.473270  7081 leader_election.cc:288] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e16f9fbc821433daee440eafa105a97 (127.0.99.131:38365), 1738567a9fd4445585dc1c48431f6fab (127.0.99.130:40217)
I20240426 13:14:41.476081  7032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1738137e13f24b1a9a1e50ab0b0f679f" candidate_uuid: "5ea4615254e241e1b041b47a1d27da61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e16f9fbc821433daee440eafa105a97" is_pre_election: true
I20240426 13:14:41.476186  7032 raft_consensus.cc:2454] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ea4615254e241e1b041b47a1d27da61 in term 0.
I20240426 13:14:41.476294  6957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1738137e13f24b1a9a1e50ab0b0f679f" candidate_uuid: "5ea4615254e241e1b041b47a1d27da61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1738567a9fd4445585dc1c48431f6fab" is_pre_election: true
I20240426 13:14:41.476411  6957 raft_consensus.cc:2454] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ea4615254e241e1b041b47a1d27da61 in term 0.
I20240426 13:14:41.476496  6845 leader_election.cc:302] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e16f9fbc821433daee440eafa105a97, 5ea4615254e241e1b041b47a1d27da61; no voters: 
I20240426 13:14:41.476694  7081 raft_consensus.cc:2781] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:14:41.476752  7081 raft_consensus.cc:482] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:14:41.476779  7081 raft_consensus.cc:3037] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:41.477406  7081 raft_consensus.cc:504] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.477682  7081 leader_election.cc:288] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [CANDIDATE]: Term 1 election: Requested vote from peers 2e16f9fbc821433daee440eafa105a97 (127.0.99.131:38365), 1738567a9fd4445585dc1c48431f6fab (127.0.99.130:40217)
I20240426 13:14:41.478005  7032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1738137e13f24b1a9a1e50ab0b0f679f" candidate_uuid: "5ea4615254e241e1b041b47a1d27da61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e16f9fbc821433daee440eafa105a97"
I20240426 13:14:41.478050  6957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1738137e13f24b1a9a1e50ab0b0f679f" candidate_uuid: "5ea4615254e241e1b041b47a1d27da61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1738567a9fd4445585dc1c48431f6fab"
I20240426 13:14:41.478119  7032 raft_consensus.cc:3037] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:41.478132  6957 raft_consensus.cc:3037] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:41.478943  7032 raft_consensus.cc:2454] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ea4615254e241e1b041b47a1d27da61 in term 1.
I20240426 13:14:41.478940  6957 raft_consensus.cc:2454] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ea4615254e241e1b041b47a1d27da61 in term 1.
I20240426 13:14:41.479228  6845 leader_election.cc:302] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e16f9fbc821433daee440eafa105a97, 5ea4615254e241e1b041b47a1d27da61; no voters: 
I20240426 13:14:41.479378  7081 raft_consensus.cc:2781] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:41.479538  7081 raft_consensus.cc:686] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 1 LEADER]: Becoming Leader. State: Replica: 5ea4615254e241e1b041b47a1d27da61, State: Running, Role: LEADER
I20240426 13:14:41.479665  7081 consensus_queue.cc:237] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } }
I20240426 13:14:41.480913  6777 catalog_manager.cc:5484] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ea4615254e241e1b041b47a1d27da61 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "5ea4615254e241e1b041b47a1d27da61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ea4615254e241e1b041b47a1d27da61" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 38777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 } health_report { overall_health: UNKNOWN } } }
I20240426 13:14:41.506968  6957 raft_consensus.cc:1261] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 1 FOLLOWER]: Refusing update from remote peer 5ea4615254e241e1b041b47a1d27da61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:41.506968  7032 raft_consensus.cc:1261] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 1 FOLLOWER]: Refusing update from remote peer 5ea4615254e241e1b041b47a1d27da61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:41.507354  7083 consensus_queue.cc:1035] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e16f9fbc821433daee440eafa105a97" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 38365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:41.507548  7081 consensus_queue.cc:1035] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1738567a9fd4445585dc1c48431f6fab" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 40217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20240426 13:14:51.519342  7090 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60204: Illegal state: Transaction hasn't been successfully started
W20240426 13:14:51.520201  7090 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60204: Illegal state: Transaction metadata for transaction 1 does not exist
I20240426 13:14:51.524376   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:14:51.528858   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:51.529009   398 tablet_replica.cc:330] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61: stopping tablet replica
I20240426 13:14:51.529104   398 raft_consensus.cc:2229] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:51.529242   398 raft_consensus.cc:2258] T 1738137e13f24b1a9a1e50ab0b0f679f P 5ea4615254e241e1b041b47a1d27da61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:51.545668   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:14:51.547250   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:14:51.550935   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:51.551079   398 tablet_replica.cc:330] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab: stopping tablet replica
I20240426 13:14:51.551185   398 raft_consensus.cc:2229] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:51.551285   398 raft_consensus.cc:2258] T 1738137e13f24b1a9a1e50ab0b0f679f P 1738567a9fd4445585dc1c48431f6fab [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:51.566609   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:14:51.568095   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:14:51.571152   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:14:51.571302   398 tablet_replica.cc:330] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97: stopping tablet replica
I20240426 13:14:51.571372   398 raft_consensus.cc:2229] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20240426 13:14:51.571442   398 raft_consensus.cc:2258] T 1738137e13f24b1a9a1e50ab0b0f679f P 2e16f9fbc821433daee440eafa105a97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:51.586887   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:14:51.588321   398 master.cc:537] Master@127.0.99.190:40671 shutting down...
I20240426 13:14:51.591154   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:14:51.591277   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:14:51.591331   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0521690ffdd34ab0ad70295db544bbc5: stopping tablet replica
I20240426 13:14:51.603180   398 master.cc:552] Master@127.0.99.190:40671 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientErrorWhenNotBegun (10361 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientRepeatCalls
I20240426 13:14:51.611615   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:39525
I20240426 13:14:51.611907   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:51.614012  7146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:51.614938  7148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:51.615183  7145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:51.616055   398 server_base.cc:1034] running on GCE node
I20240426 13:14:51.616195   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:51.616236   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:51.616250   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137291616250 us; error 0 us; skew 500 ppm
I20240426 13:14:51.617038   398 webserver.cc:460] Webserver started at http://127.0.99.190:37517/ using document root <none> and password file <none>
I20240426 13:14:51.617179   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:51.617218   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:51.617282   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:51.617659   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "d7e13655195d4f2c8c22efde57ae9b90"
format_stamp: "Formatted at 2024-04-26 13:14:51 on dist-test-slave-2jsl"
I20240426 13:14:51.618985   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:14:51.619794  7153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.620020   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:51.620083   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "d7e13655195d4f2c8c22efde57ae9b90"
format_stamp: "Formatted at 2024-04-26 13:14:51 on dist-test-slave-2jsl"
I20240426 13:14:51.620151   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:51.637045   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:51.637315   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:51.641057   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:39525
I20240426 13:14:51.648260  7205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:39525 every 8 connection(s)
I20240426 13:14:51.648644  7206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:51.650214  7206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90: Bootstrap starting.
I20240426 13:14:51.650882  7206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:51.651705  7206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90: No bootstrap required, opened a new log
I20240426 13:14:51.652015  7206 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e13655195d4f2c8c22efde57ae9b90" member_type: VOTER }
I20240426 13:14:51.652109  7206 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:51.652141  7206 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e13655195d4f2c8c22efde57ae9b90, State: Initialized, Role: FOLLOWER
I20240426 13:14:51.652248  7206 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e13655195d4f2c8c22efde57ae9b90" member_type: VOTER }
I20240426 13:14:51.652300  7206 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:14:51.652321  7206 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:14:51.652366  7206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:51.652904  7206 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e13655195d4f2c8c22efde57ae9b90" member_type: VOTER }
I20240426 13:14:51.653003  7206 leader_election.cc:302] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7e13655195d4f2c8c22efde57ae9b90; no voters: 
I20240426 13:14:51.653157  7206 leader_election.cc:288] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:14:51.653285  7209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:51.653445  7209 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 1 LEADER]: Becoming Leader. State: Replica: d7e13655195d4f2c8c22efde57ae9b90, State: Running, Role: LEADER
I20240426 13:14:51.653492  7206 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:14:51.653559  7209 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e13655195d4f2c8c22efde57ae9b90" member_type: VOTER }
I20240426 13:14:51.653919  7210 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7e13655195d4f2c8c22efde57ae9b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e13655195d4f2c8c22efde57ae9b90" member_type: VOTER } }
I20240426 13:14:51.653939  7211 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7e13655195d4f2c8c22efde57ae9b90. Latest consensus state: current_term: 1 leader_uuid: "d7e13655195d4f2c8c22efde57ae9b90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e13655195d4f2c8c22efde57ae9b90" member_type: VOTER } }
I20240426 13:14:51.654042  7211 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:51.654217  7210 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [sys.catalog]: This master's current role is: LEADER
I20240426 13:14:51.654556  7214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:14:51.655169  7214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:14:51.655354   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:14:51.656633  7214 catalog_manager.cc:1347] Generated new cluster ID: 8e89ab98cb61456c9a3a7f902ffd4c55
I20240426 13:14:51.656685  7214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:14:51.669212  7214 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:14:51.669636  7214 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:14:51.677544  7214 catalog_manager.cc:5857] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90: Generated new TSK 0
I20240426 13:14:51.677654  7214 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:14:51.687459   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:51.689582  7228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:51.689775  7227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:51.690778  7230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:51.691679   398 server_base.cc:1034] running on GCE node
I20240426 13:14:51.691828   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:51.691870   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:51.691896   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137291691895 us; error 0 us; skew 500 ppm
I20240426 13:14:51.692761   398 webserver.cc:460] Webserver started at http://127.0.99.129:38671/ using document root <none> and password file <none>
I20240426 13:14:51.692903   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:51.692952   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:51.693025   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:51.693354   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "93dc6c7c3257421c82713b53df1fc32f"
format_stamp: "Formatted at 2024-04-26 13:14:51 on dist-test-slave-2jsl"
I20240426 13:14:51.694697   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:14:51.695546  7235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.695760   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:51.695822   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "93dc6c7c3257421c82713b53df1fc32f"
format_stamp: "Formatted at 2024-04-26 13:14:51 on dist-test-slave-2jsl"
I20240426 13:14:51.695886   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:51.709942   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:51.710278   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:51.710469   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:51.710825   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:51.710860   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.710887   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:51.710919   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.714603   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:46881
I20240426 13:14:51.714627  7298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:46881 every 8 connection(s)
I20240426 13:14:51.715252   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:51.716843  7303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:51.717340  7304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:51.718226  7306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:51.719015   398 server_base.cc:1034] running on GCE node
I20240426 13:14:51.719183   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:51.719225   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:51.719249   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137291719248 us; error 0 us; skew 500 ppm
I20240426 13:14:51.719434  7299 heartbeater.cc:347] Connected to a master server at 127.0.99.190:39525
I20240426 13:14:51.719517  7299 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:51.719720  7299 heartbeater.cc:511] Master 127.0.99.190:39525 requested a full tablet report, sending...
I20240426 13:14:51.720307   398 webserver.cc:460] Webserver started at http://127.0.99.130:33003/ using document root <none> and password file <none>
I20240426 13:14:51.720392  7170 ts_manager.cc:194] Registered new tserver with Master: 93dc6c7c3257421c82713b53df1fc32f (127.0.99.129:46881)
I20240426 13:14:51.720477   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:51.720537   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:51.720613   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:51.720966   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "4641d47366b141b38a8061dd85288f21"
format_stamp: "Formatted at 2024-04-26 13:14:51 on dist-test-slave-2jsl"
I20240426 13:14:51.721092  7170 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46810
I20240426 13:14:51.722555   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:14:51.723377  7311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.723616   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:14:51.723693   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "4641d47366b141b38a8061dd85288f21"
format_stamp: "Formatted at 2024-04-26 13:14:51 on dist-test-slave-2jsl"
I20240426 13:14:51.723776   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:51.740028   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:51.740306   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:51.740550   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:51.740940   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:51.740984   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.741032   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:51.741073   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.745162   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:44729
I20240426 13:14:51.745201  7374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:44729 every 8 connection(s)
I20240426 13:14:51.745823   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:14:51.747553  7378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:51.748034  7379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:14:51.748960  7381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:14:51.749460  7375 heartbeater.cc:347] Connected to a master server at 127.0.99.190:39525
I20240426 13:14:51.749532  7375 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:51.749729  7375 heartbeater.cc:511] Master 127.0.99.190:39525 requested a full tablet report, sending...
I20240426 13:14:51.749743   398 server_base.cc:1034] running on GCE node
I20240426 13:14:51.749938   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:14:51.750057   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:14:51.750085   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137291750085 us; error 0 us; skew 500 ppm
I20240426 13:14:51.750409  7170 ts_manager.cc:194] Registered new tserver with Master: 4641d47366b141b38a8061dd85288f21 (127.0.99.130:44729)
I20240426 13:14:51.750972   398 webserver.cc:460] Webserver started at http://127.0.99.131:37899/ using document root <none> and password file <none>
I20240426 13:14:51.751015  7170 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46822
I20240426 13:14:51.751111   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:14:51.751159   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:14:51.751230   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:14:51.751576   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "ee50b4dd9ddd482fbc75e6e93819ed01"
format_stamp: "Formatted at 2024-04-26 13:14:51 on dist-test-slave-2jsl"
I20240426 13:14:51.752807   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:14:51.753638  7386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.753840   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:51.753902   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "ee50b4dd9ddd482fbc75e6e93819ed01"
format_stamp: "Formatted at 2024-04-26 13:14:51 on dist-test-slave-2jsl"
I20240426 13:14:51.754024   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRepeatCalls.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:14:51.759932   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:14:51.760282   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:14:51.760553   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:14:51.760973   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:14:51.761018   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.761065   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:14:51.761103   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:14:51.765717   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:40861
I20240426 13:14:51.765738  7449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:40861 every 8 connection(s)
I20240426 13:14:51.769379  7450 heartbeater.cc:347] Connected to a master server at 127.0.99.190:39525
I20240426 13:14:51.769449  7450 heartbeater.cc:464] Registering TS with master...
I20240426 13:14:51.769603  7450 heartbeater.cc:511] Master 127.0.99.190:39525 requested a full tablet report, sending...
I20240426 13:14:51.770145  7170 ts_manager.cc:194] Registered new tserver with Master: ee50b4dd9ddd482fbc75e6e93819ed01 (127.0.99.131:40861)
I20240426 13:14:51.770582  7170 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46826
I20240426 13:14:51.770619   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004487738s
I20240426 13:14:51.771109   398 test_util.cc:268] Using random seed: -1335931676
I20240426 13:14:51.777376  7170 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46838:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:14:51.777714  7170 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:14:51.786224  7339 tablet_service.cc:1467] Processing CreateTablet for tablet de57d97eb40245959feb8348321c8dda (DEFAULT_TABLE table=test-workload [id=09807a6ca8f44409b9963759a6a29722]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:51.786227  7414 tablet_service.cc:1467] Processing CreateTablet for tablet de57d97eb40245959feb8348321c8dda (DEFAULT_TABLE table=test-workload [id=09807a6ca8f44409b9963759a6a29722]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:51.786542  7339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de57d97eb40245959feb8348321c8dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:51.786542  7414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de57d97eb40245959feb8348321c8dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:51.786988  7263 tablet_service.cc:1467] Processing CreateTablet for tablet de57d97eb40245959feb8348321c8dda (DEFAULT_TABLE table=test-workload [id=09807a6ca8f44409b9963759a6a29722]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:14:51.787227  7263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de57d97eb40245959feb8348321c8dda. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:14:51.788487  7465 tablet_bootstrap.cc:492] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01: Bootstrap starting.
I20240426 13:14:51.789134  7466 tablet_bootstrap.cc:492] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21: Bootstrap starting.
I20240426 13:14:51.789326  7465 tablet_bootstrap.cc:654] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:51.789703  7467 tablet_bootstrap.cc:492] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f: Bootstrap starting.
I20240426 13:14:51.790037  7466 tablet_bootstrap.cc:654] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:51.790473  7465 tablet_bootstrap.cc:492] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01: No bootstrap required, opened a new log
I20240426 13:14:51.790555  7465 ts_tablet_manager.cc:1367] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:14:51.790730  7467 tablet_bootstrap.cc:654] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f: Neither blocks nor log segments found. Creating new log.
I20240426 13:14:51.790957  7465 raft_consensus.cc:348] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:51.791047  7465 raft_consensus.cc:374] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:51.791110  7465 raft_consensus.cc:729] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee50b4dd9ddd482fbc75e6e93819ed01, State: Initialized, Role: FOLLOWER
I20240426 13:14:51.791054  7466 tablet_bootstrap.cc:492] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21: No bootstrap required, opened a new log
I20240426 13:14:51.791260  7466 ts_tablet_manager.cc:1367] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:14:51.791244  7465 consensus_queue.cc:260] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:51.791535  7465 ts_tablet_manager.cc:1398] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:51.791664  7450 heartbeater.cc:503] Master 127.0.99.190:39525 was elected leader, sending a full tablet report...
I20240426 13:14:51.791743  7467 tablet_bootstrap.cc:492] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f: No bootstrap required, opened a new log
I20240426 13:14:51.791698  7466 raft_consensus.cc:348] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:51.791811  7466 raft_consensus.cc:374] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:51.791841  7467 ts_tablet_manager.cc:1367] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:14:51.791853  7466 raft_consensus.cc:729] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4641d47366b141b38a8061dd85288f21, State: Initialized, Role: FOLLOWER
I20240426 13:14:51.791970  7466 consensus_queue.cc:260] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:51.792236  7467 raft_consensus.cc:348] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:51.792302  7466 ts_tablet_manager.cc:1398] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:51.792321  7375 heartbeater.cc:503] Master 127.0.99.190:39525 was elected leader, sending a full tablet report...
I20240426 13:14:51.792332  7467 raft_consensus.cc:374] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:14:51.792402  7467 raft_consensus.cc:729] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93dc6c7c3257421c82713b53df1fc32f, State: Initialized, Role: FOLLOWER
I20240426 13:14:51.792516  7467 consensus_queue.cc:260] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:51.792842  7467 ts_tablet_manager.cc:1398] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:14:51.793223  7299 heartbeater.cc:503] Master 127.0.99.190:39525 was elected leader, sending a full tablet report...
W20240426 13:14:51.965209  7300 tablet.cc:2369] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 13:14:52.033936  7472 raft_consensus.cc:482] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:14:52.034101  7472 raft_consensus.cc:504] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:52.034621  7472 leader_election.cc:288] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee50b4dd9ddd482fbc75e6e93819ed01 (127.0.99.131:40861), 93dc6c7c3257421c82713b53df1fc32f (127.0.99.129:46881)
I20240426 13:14:52.037590  7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de57d97eb40245959feb8348321c8dda" candidate_uuid: "4641d47366b141b38a8061dd85288f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" is_pre_election: true
I20240426 13:14:52.037638  7273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de57d97eb40245959feb8348321c8dda" candidate_uuid: "4641d47366b141b38a8061dd85288f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93dc6c7c3257421c82713b53df1fc32f" is_pre_election: true
I20240426 13:14:52.037724  7424 raft_consensus.cc:2454] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4641d47366b141b38a8061dd85288f21 in term 0.
I20240426 13:14:52.037766  7273 raft_consensus.cc:2454] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4641d47366b141b38a8061dd85288f21 in term 0.
I20240426 13:14:52.038056  7315 leader_election.cc:302] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4641d47366b141b38a8061dd85288f21, ee50b4dd9ddd482fbc75e6e93819ed01; no voters: 
I20240426 13:14:52.038208  7472 raft_consensus.cc:2781] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:14:52.038269  7472 raft_consensus.cc:482] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:14:52.038341  7472 raft_consensus.cc:3037] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:52.039181  7472 raft_consensus.cc:504] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:52.039500  7472 leader_election.cc:288] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [CANDIDATE]: Term 1 election: Requested vote from peers ee50b4dd9ddd482fbc75e6e93819ed01 (127.0.99.131:40861), 93dc6c7c3257421c82713b53df1fc32f (127.0.99.129:46881)
I20240426 13:14:52.039804  7273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de57d97eb40245959feb8348321c8dda" candidate_uuid: "4641d47366b141b38a8061dd85288f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93dc6c7c3257421c82713b53df1fc32f"
I20240426 13:14:52.039804  7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de57d97eb40245959feb8348321c8dda" candidate_uuid: "4641d47366b141b38a8061dd85288f21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01"
I20240426 13:14:52.039969  7273 raft_consensus.cc:3037] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:52.039969  7424 raft_consensus.cc:3037] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:14:52.040963  7273 raft_consensus.cc:2454] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4641d47366b141b38a8061dd85288f21 in term 1.
I20240426 13:14:52.040963  7424 raft_consensus.cc:2454] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4641d47366b141b38a8061dd85288f21 in term 1.
I20240426 13:14:52.041265  7315 leader_election.cc:302] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4641d47366b141b38a8061dd85288f21, ee50b4dd9ddd482fbc75e6e93819ed01; no voters: 
I20240426 13:14:52.041399  7472 raft_consensus.cc:2781] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:14:52.041601  7472 raft_consensus.cc:686] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 1 LEADER]: Becoming Leader. State: Replica: 4641d47366b141b38a8061dd85288f21, State: Running, Role: LEADER
I20240426 13:14:52.041751  7472 consensus_queue.cc:237] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:14:52.043296  7168 catalog_manager.cc:5484] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4641d47366b141b38a8061dd85288f21 (127.0.99.130). New cstate: current_term: 1 leader_uuid: "4641d47366b141b38a8061dd85288f21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } health_report { overall_health: UNKNOWN } } }
I20240426 13:14:52.089429  7273 raft_consensus.cc:1261] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 1 FOLLOWER]: Refusing update from remote peer 4641d47366b141b38a8061dd85288f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:52.089555  7424 raft_consensus.cc:1261] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 1 FOLLOWER]: Refusing update from remote peer 4641d47366b141b38a8061dd85288f21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:14:52.089852  7475 consensus_queue.cc:1035] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:52.090003  7472 consensus_queue.cc:1035] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:14:52.093106  7482 mvcc.cc:205] Tried to move back new op lower bound from 7021106348395245568 to 7021106348204662784. Current Snapshot: MvccSnapshot[applied={T|T < 7021106348395245568}]
I20240426 13:15:02.476938  7510 raft_consensus.cc:979] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21: : Instructing follower 93dc6c7c3257421c82713b53df1fc32f to start an election
I20240426 13:15:02.477046  7510 raft_consensus.cc:1067] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 1 LEADER]: Signalling peer 93dc6c7c3257421c82713b53df1fc32f to start an election
I20240426 13:15:02.477407  7273 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "de57d97eb40245959feb8348321c8dda"
dest_uuid: "93dc6c7c3257421c82713b53df1fc32f"
 from {username='slave'} at 127.0.0.1:46260
I20240426 13:15:02.477516  7273 raft_consensus.cc:482] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:15:02.477569  7273 raft_consensus.cc:3037] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:02.478359  7273 raft_consensus.cc:504] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:15:02.478739  7273 leader_election.cc:288] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [CANDIDATE]: Term 2 election: Requested vote from peers 4641d47366b141b38a8061dd85288f21 (127.0.99.130:44729), ee50b4dd9ddd482fbc75e6e93819ed01 (127.0.99.131:40861)
I20240426 13:15:02.482610  7349 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de57d97eb40245959feb8348321c8dda" candidate_uuid: "93dc6c7c3257421c82713b53df1fc32f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3045 } } ignore_live_leader: true dest_uuid: "4641d47366b141b38a8061dd85288f21"
I20240426 13:15:02.482630  7424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de57d97eb40245959feb8348321c8dda" candidate_uuid: "93dc6c7c3257421c82713b53df1fc32f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3045 } } ignore_live_leader: true dest_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01"
I20240426 13:15:02.482695  7349 raft_consensus.cc:3032] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:15:02.482730  7424 raft_consensus.cc:3037] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:02.482734  7349 raft_consensus.cc:729] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4641d47366b141b38a8061dd85288f21, State: Running, Role: LEADER
I20240426 13:15:02.482878  7349 consensus_queue.cc:260] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3045, Committed index: 3045, Last appended: 1.3045, Last appended by leader: 3045, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:15:02.483011  7349 raft_consensus.cc:3037] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:02.483464  7424 raft_consensus.cc:2454] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93dc6c7c3257421c82713b53df1fc32f in term 2.
I20240426 13:15:02.483668  7349 raft_consensus.cc:2454] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93dc6c7c3257421c82713b53df1fc32f in term 2.
I20240426 13:15:02.483747  7239 leader_election.cc:302] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93dc6c7c3257421c82713b53df1fc32f, ee50b4dd9ddd482fbc75e6e93819ed01; no voters: 
I20240426 13:15:02.484098  7526 raft_consensus.cc:2781] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:15:02.484252  7526 raft_consensus.cc:686] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 2 LEADER]: Becoming Leader. State: Replica: 93dc6c7c3257421c82713b53df1fc32f, State: Running, Role: LEADER
I20240426 13:15:02.484373  7526 consensus_queue.cc:237] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3045, Committed index: 3045, Last appended: 1.3045, Last appended by leader: 3045, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } }
I20240426 13:15:02.485498  7168 catalog_manager.cc:5484] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f reported cstate change: term changed from 1 to 2, leader changed from 4641d47366b141b38a8061dd85288f21 (127.0.99.130) to 93dc6c7c3257421c82713b53df1fc32f (127.0.99.129). New cstate: current_term: 2 leader_uuid: "93dc6c7c3257421c82713b53df1fc32f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93dc6c7c3257421c82713b53df1fc32f" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46881 } health_report { overall_health: HEALTHY } } }
I20240426 13:15:02.501269  7349 raft_consensus.cc:1261] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 2 FOLLOWER]: Refusing update from remote peer 93dc6c7c3257421c82713b53df1fc32f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3045. Preceding OpId from leader: term: 2 index: 3047. (index mismatch)
I20240426 13:15:02.501284  7424 raft_consensus.cc:1261] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 2 FOLLOWER]: Refusing update from remote peer 93dc6c7c3257421c82713b53df1fc32f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3045. Preceding OpId from leader: term: 2 index: 3047. (index mismatch)
I20240426 13:15:02.501588  7526 consensus_queue.cc:1035] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4641d47366b141b38a8061dd85288f21" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
I20240426 13:15:02.501705  7527 consensus_queue.cc:1035] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee50b4dd9ddd482fbc75e6e93819ed01" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 40861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
W20240426 13:15:02.504958  7484 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34728: Illegal state: Transaction 1 already open
W20240426 13:15:02.508520  7484 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34728: Illegal state: Transaction 1 commit already in progress
W20240426 13:15:02.511837  7484 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34728: Illegal state: Transaction metadata for transaction 1 already set
W20240426 13:15:02.515329  7484 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34728: Illegal state: Transaction 2 already open
W20240426 13:15:02.518469  7484 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34728: Illegal state: Transaction 2 commit already in progress
W20240426 13:15:02.521708  7484 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:34728: Illegal state: Transaction metadata for transaction 2 already set
I20240426 13:15:02.522503   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:15:02.526012   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:02.526136   398 tablet_replica.cc:330] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f: stopping tablet replica
I20240426 13:15:02.526243   398 raft_consensus.cc:2229] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:15:02.526378   398 raft_consensus.cc:2258] T de57d97eb40245959feb8348321c8dda P 93dc6c7c3257421c82713b53df1fc32f [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:02.532099   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:15:02.533540   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:15:02.537077   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:02.537236   398 tablet_replica.cc:330] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21: stopping tablet replica
I20240426 13:15:02.537322   398 raft_consensus.cc:2229] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:15:02.537400   398 raft_consensus.cc:2258] T de57d97eb40245959feb8348321c8dda P 4641d47366b141b38a8061dd85288f21 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:02.552989   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:15:02.554646   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:15:02.558420   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:02.558545   398 tablet_replica.cc:330] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01: stopping tablet replica
I20240426 13:15:02.558635   398 raft_consensus.cc:2229] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:15:02.558706   398 raft_consensus.cc:2258] T de57d97eb40245959feb8348321c8dda P ee50b4dd9ddd482fbc75e6e93819ed01 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:02.563741   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:15:02.564996   398 master.cc:537] Master@127.0.99.190:39525 shutting down...
I20240426 13:15:02.567515   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:15:02.567605   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:02.567658   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P d7e13655195d4f2c8c22efde57ae9b90: stopping tablet replica
I20240426 13:15:02.579496   398 master.cc:552] Master@127.0.99.190:39525 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientRepeatCalls (10975 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority
I20240426 13:15:02.587428   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:36345
I20240426 13:15:02.587692   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:02.589141  7542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:02.590026  7545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:02.590812  7543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:02.591640   398 server_base.cc:1034] running on GCE node
I20240426 13:15:02.591743   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:02.591775   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:02.591795   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137302591795 us; error 0 us; skew 500 ppm
I20240426 13:15:02.592556   398 webserver.cc:460] Webserver started at http://127.0.99.190:36841/ using document root <none> and password file <none>
I20240426 13:15:02.592666   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:02.592703   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:02.592761   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:02.593070   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "24075458728e449f9a6f38cb2518744b"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:02.594204   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:15:02.594892  7550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.595104   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:15:02.595160   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "24075458728e449f9a6f38cb2518744b"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:02.595214   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:02.604202   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:02.604445   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:02.607735   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:36345
I20240426 13:15:02.611275  7602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:36345 every 8 connection(s)
I20240426 13:15:02.612931  7603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:02.615307  7603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b: Bootstrap starting.
I20240426 13:15:02.615877  7603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:02.616771  7603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b: No bootstrap required, opened a new log
I20240426 13:15:02.617084  7603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24075458728e449f9a6f38cb2518744b" member_type: VOTER }
I20240426 13:15:02.617153  7603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:02.617192  7603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24075458728e449f9a6f38cb2518744b, State: Initialized, Role: FOLLOWER
I20240426 13:15:02.617285  7603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24075458728e449f9a6f38cb2518744b" member_type: VOTER }
I20240426 13:15:02.617355  7603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:15:02.617383  7603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:15:02.617424  7603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:02.617915  7603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24075458728e449f9a6f38cb2518744b" member_type: VOTER }
I20240426 13:15:02.618043  7603 leader_election.cc:302] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 24075458728e449f9a6f38cb2518744b; no voters: 
I20240426 13:15:02.618167  7603 leader_election.cc:288] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:15:02.618239  7606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:15:02.618424  7606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 1 LEADER]: Becoming Leader. State: Replica: 24075458728e449f9a6f38cb2518744b, State: Running, Role: LEADER
I20240426 13:15:02.618517  7603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:15:02.618538  7606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24075458728e449f9a6f38cb2518744b" member_type: VOTER }
I20240426 13:15:02.618866  7607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "24075458728e449f9a6f38cb2518744b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24075458728e449f9a6f38cb2518744b" member_type: VOTER } }
I20240426 13:15:02.618964  7607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [sys.catalog]: This master's current role is: LEADER
I20240426 13:15:02.618877  7608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 24075458728e449f9a6f38cb2518744b. Latest consensus state: current_term: 1 leader_uuid: "24075458728e449f9a6f38cb2518744b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "24075458728e449f9a6f38cb2518744b" member_type: VOTER } }
I20240426 13:15:02.619060  7608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [sys.catalog]: This master's current role is: LEADER
I20240426 13:15:02.619477  7612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:15:02.620100  7612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:15:02.620357   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:15:02.621614  7612 catalog_manager.cc:1347] Generated new cluster ID: 3d78453d984346ca9fe331241a99ab61
I20240426 13:15:02.621660  7612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:15:02.644047  7612 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:15:02.644480  7612 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:15:02.657313  7612 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b: Generated new TSK 0
I20240426 13:15:02.657429  7612 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:15:02.684465   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:02.686198  7624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:02.687125  7625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:02.687968  7627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:02.688815   398 server_base.cc:1034] running on GCE node
I20240426 13:15:02.688930   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:02.688966   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:02.688979   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137302688979 us; error 0 us; skew 500 ppm
I20240426 13:15:02.689710   398 webserver.cc:460] Webserver started at http://127.0.99.129:38651/ using document root <none> and password file <none>
I20240426 13:15:02.689826   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:02.689864   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:02.689924   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:02.690258   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "e125aefc176d43b8b20ebc337619cac9"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:02.691744   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:15:02.692620  7632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.692816   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:02.692869   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "e125aefc176d43b8b20ebc337619cac9"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:02.692945   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:02.703621   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:02.703877   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:02.704105   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:02.704468   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:02.704504   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.704532   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:02.704562   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.708220   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:37529
I20240426 13:15:02.708240  7695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:37529 every 8 connection(s)
I20240426 13:15:02.708860   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:02.710398  7700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:02.711159  7703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:02.711854  7701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:02.712658   398 server_base.cc:1034] running on GCE node
I20240426 13:15:02.712857   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:02.712929   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:02.712957   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137302712956 us; error 0 us; skew 500 ppm
I20240426 13:15:02.712972  7696 heartbeater.cc:347] Connected to a master server at 127.0.99.190:36345
I20240426 13:15:02.713032  7696 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:02.713215  7696 heartbeater.cc:511] Master 127.0.99.190:36345 requested a full tablet report, sending...
I20240426 13:15:02.713698   398 webserver.cc:460] Webserver started at http://127.0.99.130:36393/ using document root <none> and password file <none>
I20240426 13:15:02.713728  7567 ts_manager.cc:194] Registered new tserver with Master: e125aefc176d43b8b20ebc337619cac9 (127.0.99.129:37529)
I20240426 13:15:02.713804   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:02.713852   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:02.713919   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:02.714246   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "60c1069786264e8698e4d4a42cfc1004"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:02.714428  7567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56190
I20240426 13:15:02.715495   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:02.716233  7708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.716449   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:02.716513   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "60c1069786264e8698e4d4a42cfc1004"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:02.716573   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:02.723641   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:02.723886   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:02.724112   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:02.724460   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:02.724493   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.724530   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:02.724560   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.727901   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:41149
I20240426 13:15:02.727959  7771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:41149 every 8 connection(s)
I20240426 13:15:02.728523   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:02.730094  7775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:02.730463  7776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:02.731263   398 server_base.cc:1034] running on GCE node
W20240426 13:15:02.731979  7778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:02.732170  7772 heartbeater.cc:347] Connected to a master server at 127.0.99.190:36345
I20240426 13:15:02.732266  7772 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:02.732292   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:02.732342   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:02.732376   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137302732376 us; error 0 us; skew 500 ppm
I20240426 13:15:02.732448  7772 heartbeater.cc:511] Master 127.0.99.190:36345 requested a full tablet report, sending...
I20240426 13:15:02.733024  7567 ts_manager.cc:194] Registered new tserver with Master: 60c1069786264e8698e4d4a42cfc1004 (127.0.99.130:41149)
I20240426 13:15:02.733151   398 webserver.cc:460] Webserver started at http://127.0.99.131:45441/ using document root <none> and password file <none>
I20240426 13:15:02.733271   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:02.733311   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:02.733371   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:02.733546  7567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56194
I20240426 13:15:02.733659   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "431afc32eada417fa8920976b58db136"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:02.735136   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:15:02.735896  7783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.736116   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:02.736173   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "431afc32eada417fa8920976b58db136"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:02.736232   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:02.748711   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:02.748983   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:02.749217   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:02.749558   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:02.749594   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.749636   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:02.749662   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:02.754041  7846 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:41641 every 8 connection(s)
I20240426 13:15:02.754046   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:41641
I20240426 13:15:02.757416  7847 heartbeater.cc:347] Connected to a master server at 127.0.99.190:36345
I20240426 13:15:02.757486  7847 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:02.757606  7847 heartbeater.cc:511] Master 127.0.99.190:36345 requested a full tablet report, sending...
I20240426 13:15:02.758098  7567 ts_manager.cc:194] Registered new tserver with Master: 431afc32eada417fa8920976b58db136 (127.0.99.131:41641)
I20240426 13:15:02.758464  7567 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56210
I20240426 13:15:02.758893   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004502424s
I20240426 13:15:02.759284   398 test_util.cc:268] Using random seed: -1324943501
I20240426 13:15:02.765132  7567 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56222:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:15:02.765432  7567 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:15:02.773155  7660 tablet_service.cc:1467] Processing CreateTablet for tablet 51587a1b2ea04817b4ecfc2b137f5ac0 (DEFAULT_TABLE table=test-workload [id=b817282ad2dc4a7aa36bb2e74187f754]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:02.773309  7736 tablet_service.cc:1467] Processing CreateTablet for tablet 51587a1b2ea04817b4ecfc2b137f5ac0 (DEFAULT_TABLE table=test-workload [id=b817282ad2dc4a7aa36bb2e74187f754]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:02.773396  7660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51587a1b2ea04817b4ecfc2b137f5ac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:02.773425  7811 tablet_service.cc:1467] Processing CreateTablet for tablet 51587a1b2ea04817b4ecfc2b137f5ac0 (DEFAULT_TABLE table=test-workload [id=b817282ad2dc4a7aa36bb2e74187f754]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:02.773481  7736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51587a1b2ea04817b4ecfc2b137f5ac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:02.773597  7811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51587a1b2ea04817b4ecfc2b137f5ac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:02.775164  7862 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Bootstrap starting.
I20240426 13:15:02.775437  7863 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Bootstrap starting.
I20240426 13:15:02.775979  7864 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9: Bootstrap starting.
I20240426 13:15:02.776047  7862 tablet_bootstrap.cc:654] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:02.776311  7863 tablet_bootstrap.cc:654] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:02.776687  7864 tablet_bootstrap.cc:654] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:02.777166  7862 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: No bootstrap required, opened a new log
I20240426 13:15:02.777217  7863 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: No bootstrap required, opened a new log
I20240426 13:15:02.777251  7862 ts_tablet_manager.cc:1367] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:15:02.777269  7863 ts_tablet_manager.cc:1367] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:15:02.777596  7864 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9: No bootstrap required, opened a new log
I20240426 13:15:02.777674  7864 ts_tablet_manager.cc:1367] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:15:02.777622  7862 raft_consensus.cc:348] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.777706  7863 raft_consensus.cc:348] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.777771  7862 raft_consensus.cc:374] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:02.777809  7862 raft_consensus.cc:729] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c1069786264e8698e4d4a42cfc1004, State: Initialized, Role: FOLLOWER
I20240426 13:15:02.777807  7863 raft_consensus.cc:374] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:02.777901  7863 raft_consensus.cc:729] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431afc32eada417fa8920976b58db136, State: Initialized, Role: FOLLOWER
I20240426 13:15:02.778018  7863 consensus_queue.cc:260] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.778021  7862 consensus_queue.cc:260] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.778087  7864 raft_consensus.cc:348] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.778224  7864 raft_consensus.cc:374] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:02.778302  7864 raft_consensus.cc:729] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e125aefc176d43b8b20ebc337619cac9, State: Initialized, Role: FOLLOWER
I20240426 13:15:02.778432  7847 heartbeater.cc:503] Master 127.0.99.190:36345 was elected leader, sending a full tablet report...
I20240426 13:15:02.778577  7772 heartbeater.cc:503] Master 127.0.99.190:36345 was elected leader, sending a full tablet report...
I20240426 13:15:02.778548  7864 consensus_queue.cc:260] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.778403  7862 ts_tablet_manager.cc:1398] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:02.778403  7863 ts_tablet_manager.cc:1398] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:15:02.779363  7696 heartbeater.cc:503] Master 127.0.99.190:36345 was elected leader, sending a full tablet report...
I20240426 13:15:02.779377  7864 ts_tablet_manager.cc:1398] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9: Time spent starting tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:15:02.836325  7868 raft_consensus.cc:482] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:15:02.836434  7868 raft_consensus.cc:504] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.836834  7868 leader_election.cc:288] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e125aefc176d43b8b20ebc337619cac9 (127.0.99.129:37529), 60c1069786264e8698e4d4a42cfc1004 (127.0.99.130:41149)
I20240426 13:15:02.840240  7670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51587a1b2ea04817b4ecfc2b137f5ac0" candidate_uuid: "431afc32eada417fa8920976b58db136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e125aefc176d43b8b20ebc337619cac9" is_pre_election: true
I20240426 13:15:02.840350  7670 raft_consensus.cc:2454] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 431afc32eada417fa8920976b58db136 in term 0.
I20240426 13:15:02.840514  7746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51587a1b2ea04817b4ecfc2b137f5ac0" candidate_uuid: "431afc32eada417fa8920976b58db136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60c1069786264e8698e4d4a42cfc1004" is_pre_election: true
I20240426 13:15:02.840626  7746 raft_consensus.cc:2454] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 431afc32eada417fa8920976b58db136 in term 0.
I20240426 13:15:02.840639  7786 leader_election.cc:302] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431afc32eada417fa8920976b58db136, e125aefc176d43b8b20ebc337619cac9; no voters: 
I20240426 13:15:02.840809  7868 raft_consensus.cc:2781] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:15:02.840885  7868 raft_consensus.cc:482] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:15:02.840924  7868 raft_consensus.cc:3037] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:02.841702  7868 raft_consensus.cc:504] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.842017  7868 leader_election.cc:288] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [CANDIDATE]: Term 1 election: Requested vote from peers e125aefc176d43b8b20ebc337619cac9 (127.0.99.129:37529), 60c1069786264e8698e4d4a42cfc1004 (127.0.99.130:41149)
I20240426 13:15:02.842265  7670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51587a1b2ea04817b4ecfc2b137f5ac0" candidate_uuid: "431afc32eada417fa8920976b58db136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e125aefc176d43b8b20ebc337619cac9"
I20240426 13:15:02.842265  7746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51587a1b2ea04817b4ecfc2b137f5ac0" candidate_uuid: "431afc32eada417fa8920976b58db136" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60c1069786264e8698e4d4a42cfc1004"
I20240426 13:15:02.842407  7746 raft_consensus.cc:3037] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:02.842407  7670 raft_consensus.cc:3037] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:02.843317  7670 raft_consensus.cc:2454] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 431afc32eada417fa8920976b58db136 in term 1.
I20240426 13:15:02.843317  7746 raft_consensus.cc:2454] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 431afc32eada417fa8920976b58db136 in term 1.
I20240426 13:15:02.843582  7786 leader_election.cc:302] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431afc32eada417fa8920976b58db136, e125aefc176d43b8b20ebc337619cac9; no voters: 
I20240426 13:15:02.843711  7868 raft_consensus.cc:2781] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:15:02.843896  7868 raft_consensus.cc:686] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 1 LEADER]: Becoming Leader. State: Replica: 431afc32eada417fa8920976b58db136, State: Running, Role: LEADER
I20240426 13:15:02.844038  7868 consensus_queue.cc:237] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:02.845455  7566 catalog_manager.cc:5484] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 reported cstate change: term changed from 0 to 1, leader changed from <none> to 431afc32eada417fa8920976b58db136 (127.0.99.131). New cstate: current_term: 1 leader_uuid: "431afc32eada417fa8920976b58db136" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } health_report { overall_health: UNKNOWN } } }
I20240426 13:15:02.886435  7670 raft_consensus.cc:1261] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 1 FOLLOWER]: Refusing update from remote peer 431afc32eada417fa8920976b58db136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:15:02.886436  7746 raft_consensus.cc:1261] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 1 FOLLOWER]: Refusing update from remote peer 431afc32eada417fa8920976b58db136: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:15:02.886832  7872 consensus_queue.cc:1035] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:15:02.886936  7868 consensus_queue.cc:1035] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:15:13.264720  7897 raft_consensus.cc:979] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: : Instructing follower e125aefc176d43b8b20ebc337619cac9 to start an election
I20240426 13:15:13.264819  7897 raft_consensus.cc:1067] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 1 LEADER]: Signalling peer e125aefc176d43b8b20ebc337619cac9 to start an election
I20240426 13:15:13.265223  7669 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "51587a1b2ea04817b4ecfc2b137f5ac0"
dest_uuid: "e125aefc176d43b8b20ebc337619cac9"
 from {username='slave'} at 127.0.0.1:40612
I20240426 13:15:13.265347  7669 raft_consensus.cc:482] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:15:13.265388  7669 raft_consensus.cc:3037] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:13.266172  7669 raft_consensus.cc:504] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:13.266561  7669 leader_election.cc:288] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [CANDIDATE]: Term 2 election: Requested vote from peers 431afc32eada417fa8920976b58db136 (127.0.99.131:41641), 60c1069786264e8698e4d4a42cfc1004 (127.0.99.130:41149)
I20240426 13:15:13.269646  7821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51587a1b2ea04817b4ecfc2b137f5ac0" candidate_uuid: "e125aefc176d43b8b20ebc337619cac9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3245 } } ignore_live_leader: true dest_uuid: "431afc32eada417fa8920976b58db136"
I20240426 13:15:13.269750  7821 raft_consensus.cc:3032] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:15:13.269793  7745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51587a1b2ea04817b4ecfc2b137f5ac0" candidate_uuid: "e125aefc176d43b8b20ebc337619cac9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3245 } } ignore_live_leader: true dest_uuid: "60c1069786264e8698e4d4a42cfc1004"
I20240426 13:15:13.269802  7821 raft_consensus.cc:729] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 431afc32eada417fa8920976b58db136, State: Running, Role: LEADER
I20240426 13:15:13.269891  7745 raft_consensus.cc:3037] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:13.269996  7821 consensus_queue.cc:260] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3245, Committed index: 3245, Last appended: 1.3245, Last appended by leader: 3245, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:13.270179  7821 raft_consensus.cc:3037] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:13.270900  7745 raft_consensus.cc:2454] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e125aefc176d43b8b20ebc337619cac9 in term 2.
I20240426 13:15:13.271152  7821 raft_consensus.cc:2454] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e125aefc176d43b8b20ebc337619cac9 in term 2.
I20240426 13:15:13.271204  7635 leader_election.cc:302] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60c1069786264e8698e4d4a42cfc1004, e125aefc176d43b8b20ebc337619cac9; no voters: 
I20240426 13:15:13.271464  7918 raft_consensus.cc:2781] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:15:13.271638  7918 raft_consensus.cc:686] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 2 LEADER]: Becoming Leader. State: Replica: e125aefc176d43b8b20ebc337619cac9, State: Running, Role: LEADER
I20240426 13:15:13.271773  7918 consensus_queue.cc:237] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3245, Committed index: 3245, Last appended: 1.3245, Last appended by leader: 3245, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:13.273033  7563 catalog_manager.cc:5484] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 reported cstate change: term changed from 1 to 2, leader changed from 431afc32eada417fa8920976b58db136 (127.0.99.131) to e125aefc176d43b8b20ebc337619cac9 (127.0.99.129). New cstate: current_term: 2 leader_uuid: "e125aefc176d43b8b20ebc337619cac9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } health_report { overall_health: UNKNOWN } } }
I20240426 13:15:13.279100   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:15:13.282238   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:13.282388   398 tablet_replica.cc:330] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: stopping tablet replica
I20240426 13:15:13.282486   398 raft_consensus.cc:2229] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:15:13.282624   398 raft_consensus.cc:2258] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:13.289268   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:15:13.290764   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:15:13.294747   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:13.294926   398 tablet_replica.cc:330] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: stopping tablet replica
I20240426 13:15:13.295004   398 raft_consensus.cc:2229] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:15:13.295128   398 raft_consensus.cc:2258] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:13.311125   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
W20240426 13:15:13.321802  7635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.130:41149: connect: Connection refused (error 111)
W20240426 13:15:13.322430  7635 consensus_peers.cc:485] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 -> Peer 60c1069786264e8698e4d4a42cfc1004 (127.0.99.130:41149): Couldn't send request to peer 60c1069786264e8698e4d4a42cfc1004. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.130:41149: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240426 13:15:13.322505  7635 consensus_peers.cc:485] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 -> Peer 431afc32eada417fa8920976b58db136 (127.0.99.131:41641): Couldn't send request to peer 431afc32eada417fa8920976b58db136. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.131:41641: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20240426 13:15:14.318876  7926 meta_cache.cc:302] tablet 51587a1b2ea04817b4ecfc2b137f5ac0: replica e125aefc176d43b8b20ebc337619cac9 (127.0.99.129:37529) has failed: Timed out: ParticipateInTransaction RPC to 127.0.99.129:37529 timed out after 0.999s (SENT)
I20240426 13:15:14.319422   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:14.322242  7936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:14.322517  7935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:14.323402   398 server_base.cc:1034] running on GCE node
W20240426 13:15:14.324168  7938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:14.324330   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:14.324373   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:14.324393   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137314324392 us; error 0 us; skew 500 ppm
I20240426 13:15:14.325230   398 webserver.cc:460] Webserver started at http://127.0.99.130:36393/ using document root <none> and password file <none>
I20240426 13:15:14.325364   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:14.325400   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:15:14.326326   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:14.326988  7943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:14.327194   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:14.327243   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "60c1069786264e8698e4d4a42cfc1004"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:14.327301   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:14.335966   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:14.336210   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:14.336395   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:14.336892  7950 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240426 13:15:14.337780   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:15:14.337821   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:14.337849   398 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240426 13:15:14.338506   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:15:14.338541   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:14.338603  7950 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Bootstrap starting.
I20240426 13:15:14.343003   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:41149
I20240426 13:15:14.343988  8008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:41149 every 8 connection(s)
I20240426 13:15:14.349776  8009 heartbeater.cc:347] Connected to a master server at 127.0.99.190:36345
I20240426 13:15:14.349855  8009 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:14.350059  8009 heartbeater.cc:511] Master 127.0.99.190:36345 requested a full tablet report, sending...
I20240426 13:15:14.350760  7563 ts_manager.cc:194] Re-registered known tserver with Master: 60c1069786264e8698e4d4a42cfc1004 (127.0.99.130:41149)
I20240426 13:15:14.351444  7563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40292
I20240426 13:15:15.352262  8009 heartbeater.cc:503] Master 127.0.99.190:36345 was elected leader, sending a full tablet report...
W20240426 13:15:15.767020  7635 consensus_peers.cc:485] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 -> Peer 431afc32eada417fa8920976b58db136 (127.0.99.131:41641): Couldn't send request to peer 431afc32eada417fa8920976b58db136. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.131:41641: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20240426 13:15:15.796514  7635 consensus_peers.cc:485] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 -> Peer 60c1069786264e8698e4d4a42cfc1004 (127.0.99.130:41149): Couldn't send request to peer 60c1069786264e8698e4d4a42cfc1004. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20240426 13:15:18.272753  7635 consensus_peers.cc:485] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 -> Peer 60c1069786264e8698e4d4a42cfc1004 (127.0.99.130:41149): Couldn't send request to peer 60c1069786264e8698e4d4a42cfc1004. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
W20240426 13:15:18.489575  7635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.131:41641: connect: Connection refused (error 111) [suppressed 11 similar messages]
W20240426 13:15:18.490454  7635 consensus_peers.cc:485] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 -> Peer 431afc32eada417fa8920976b58db136 (127.0.99.131:41641): Couldn't send request to peer 431afc32eada417fa8920976b58db136. Status: Network error: Client connection negotiation failed: client connection to 127.0.99.131:41641: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20240426 13:15:18.625275  7950 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Bootstrap replayed 1/1 log segments. Stats: ops{read=3245 overwritten=0 applied=3245 ignored=0} inserts{seen=162200 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:15:18.625577  7950 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Bootstrap complete.
I20240426 13:15:18.626695  7950 ts_tablet_manager.cc:1367] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Time spent bootstrapping tablet: real 4.288s	user 4.198s	sys 0.088s
I20240426 13:15:18.627033  7950 raft_consensus.cc:348] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:18.627161  7950 raft_consensus.cc:729] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c1069786264e8698e4d4a42cfc1004, State: Initialized, Role: FOLLOWER
I20240426 13:15:18.627267  7950 consensus_queue.cc:260] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3245, Last appended: 1.3245, Last appended by leader: 3245, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:18.627532  7950 ts_tablet_manager.cc:1398] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:18.627869   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:18.629742  8026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:18.630697  8029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:18.631461  8027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:18.632283   398 server_base.cc:1034] running on GCE node
I20240426 13:15:18.632405   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:18.632437   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:18.632450   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137318632450 us; error 0 us; skew 500 ppm
I20240426 13:15:18.638499   398 webserver.cc:460] Webserver started at http://127.0.99.131:45441/ using document root <none> and password file <none>
I20240426 13:15:18.638638   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:18.638667   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:15:18.639503   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:18.640210  8034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:18.640451   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:18.640507   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "431afc32eada417fa8920976b58db136"
format_stamp: "Formatted at 2024-04-26 13:15:02 on dist-test-slave-2jsl"
I20240426 13:15:18.640561   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:18.647296   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:18.647524   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:18.647727   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:18.648205  8041 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240426 13:15:18.649020   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:15:18.649061   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:18.649107   398 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240426 13:15:18.649653   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:15:18.649686   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:18.649731  8041 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Bootstrap starting.
I20240426 13:15:18.654234   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:41641
I20240426 13:15:18.654264  8099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:41641 every 8 connection(s)
I20240426 13:15:18.667882  8100 heartbeater.cc:347] Connected to a master server at 127.0.99.190:36345
I20240426 13:15:18.667956  8100 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:18.668167  8100 heartbeater.cc:511] Master 127.0.99.190:36345 requested a full tablet report, sending...
I20240426 13:15:18.668753  7563 ts_manager.cc:194] Re-registered known tserver with Master: 431afc32eada417fa8920976b58db136 (127.0.99.131:41641)
I20240426 13:15:18.669354  7563 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40308
W20240426 13:15:18.772799  8109 rpcz_store.cc:267] Call kudu.tserver.TabletServerAdminService.ParticipateInTransaction from 127.0.0.1:39114 (ReqId={client: 2e85c92d8e054d5d9d42b730c0f489dd, seq_no=0, attempt_no=0}) took 5451 ms (client timeout 999 ms). Trace:
W20240426 13:15:18.772977  8109 rpcz_store.cc:269] 0426 13:15:13.320965 (+     0us) service_pool.cc:168] Inserting onto call queue
0426 13:15:13.321008 (+    43us) service_pool.cc:225] Handling call
0426 13:15:18.772742 (+5451734us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0426 13:15:13.321175 (+     0us) participant_op.cc:163] PREPARE: starting
0426 13:15:13.321198 (+    23us) participant_op.cc:199] PREPARE: finished
0426 13:15:13.321213 (+    15us) participant_op.cc:212] START. Timestamp: 1714137313321205
0426 13:15:13.321214 (+     1us) op_driver.cc:331] REPLICATION: starting
0426 13:15:13.321290 (+    76us) log.cc:842] Serialized 66 byte log entry
0426 13:15:18.771704 (+5450414us) op_driver.cc:444] REPLICATION: finished
0426 13:15:18.772544 (+   840us) participant_op.cc:265] APPLY: starting
0426 13:15:18.772677 (+   133us) participant_op.cc:270] APPLY: finished
0426 13:15:18.772701 (+    24us) log.cc:842] Serialized 15 byte log entry
0426 13:15:18.772726 (+    25us) participant_op.cc:94] Released txn lock
0426 13:15:18.772730 (+     4us) participant_op.cc:291] FINISH: Op applied
Metrics: {"child_traces":[["op",{"apply.queue_time_us":799,"prepare.queue_time_us":33,"prepare.run_cpu_time_us":174,"prepare.run_wall_time_us":174,"raft.queue_time_us":62,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":255,"replication_time_us":5450471,"spinlock_wait_cycles":23424,"thread_start_us":71,"threads_started":1}]]}
I20240426 13:15:19.670073  8100 heartbeater.cc:503] Master 127.0.99.190:36345 was elected leader, sending a full tablet report...
W20240426 13:15:21.006886  7635 consensus_peers.cc:485] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 -> Peer 431afc32eada417fa8920976b58db136 (127.0.99.131:41641): Couldn't send request to peer 431afc32eada417fa8920976b58db136. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
I20240426 13:15:23.059942  8041 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Bootstrap replayed 1/1 log segments. Stats: ops{read=3245 overwritten=0 applied=3245 ignored=0} inserts{seen=162200 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:15:23.060284  8041 tablet_bootstrap.cc:492] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Bootstrap complete.
I20240426 13:15:23.061429  8041 ts_tablet_manager.cc:1367] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Time spent bootstrapping tablet: real 4.412s	user 4.318s	sys 0.088s
I20240426 13:15:23.061776  8041 raft_consensus.cc:348] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:23.061872  8041 raft_consensus.cc:729] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431afc32eada417fa8920976b58db136, State: Initialized, Role: FOLLOWER
I20240426 13:15:23.062016  8041 consensus_queue.cc:260] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3245, Last appended: 1.3245, Last appended by leader: 3245, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431afc32eada417fa8920976b58db136" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41641 } } peers { permanent_uuid: "e125aefc176d43b8b20ebc337619cac9" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37529 } } peers { permanent_uuid: "60c1069786264e8698e4d4a42cfc1004" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41149 } }
I20240426 13:15:23.062294  8041 ts_tablet_manager.cc:1398] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136: Time spent starting tablet: real 0.001s	user 0.004s	sys 0.000s
I20240426 13:15:23.094517   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:15:23.097610   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:23.097775   398 tablet_replica.cc:330] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9: stopping tablet replica
I20240426 13:15:23.097887   398 raft_consensus.cc:2229] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:15:23.098073   398 raft_consensus.cc:2258] T 51587a1b2ea04817b4ecfc2b137f5ac0 P e125aefc176d43b8b20ebc337619cac9 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:23.113591   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:15:23.115190   398 tablet_server.cc:178] TabletServer@127.0.99.130:41149 shutting down...
I20240426 13:15:23.118433   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:23.118562   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:15:23.118646   398 raft_consensus.cc:2229] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:15:23.118728   398 raft_consensus.cc:2258] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 60c1069786264e8698e4d4a42cfc1004 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:23.133756   398 tablet_server.cc:194] TabletServer@127.0.99.130:41149 shutdown complete.
I20240426 13:15:23.135308   398 tablet_server.cc:178] TabletServer@127.0.99.131:41641 shutting down...
I20240426 13:15:23.138064   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:23.138267   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:15:23.138376   398 raft_consensus.cc:2229] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 2 FOLLOWER]: Raft consensus shutting down.
I20240426 13:15:23.138469   398 raft_consensus.cc:2258] T 51587a1b2ea04817b4ecfc2b137f5ac0 P 431afc32eada417fa8920976b58db136 [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:23.153636   398 tablet_server.cc:194] TabletServer@127.0.99.131:41641 shutdown complete.
I20240426 13:15:23.155158   398 master.cc:537] Master@127.0.99.190:36345 shutting down...
I20240426 13:15:23.157763   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:15:23.157871   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:23.157928   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 24075458728e449f9a6f38cb2518744b: stopping tablet replica
I20240426 13:15:23.169693   398 master.cc:552] Master@127.0.99.190:36345 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientTimeoutWhenNoMajority (20590 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap
I20240426 13:15:23.178277   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:37749
I20240426 13:15:23.178536   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:23.180192  8117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:23.181100  8120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:23.181936  8118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:23.182684   398 server_base.cc:1034] running on GCE node
I20240426 13:15:23.182796   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:23.182825   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:23.182838   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137323182838 us; error 0 us; skew 500 ppm
I20240426 13:15:23.183674   398 webserver.cc:460] Webserver started at http://127.0.99.190:33881/ using document root <none> and password file <none>
I20240426 13:15:23.183789   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:23.183825   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:23.183876   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:23.184211   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "5272f02dc6d4409f8d4754df958c9aeb"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:23.185462   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:23.186251  8125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.186601   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:23.186653   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "5272f02dc6d4409f8d4754df958c9aeb"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:23.186710   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:23.207624   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:23.207845   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:23.211225   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:37749
I20240426 13:15:23.218600  8177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:37749 every 8 connection(s)
I20240426 13:15:23.218942  8178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:23.220394  8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb: Bootstrap starting.
I20240426 13:15:23.221011  8178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:23.221820  8178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb: No bootstrap required, opened a new log
I20240426 13:15:23.222141  8178 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5272f02dc6d4409f8d4754df958c9aeb" member_type: VOTER }
I20240426 13:15:23.222218  8178 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:23.222251  8178 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5272f02dc6d4409f8d4754df958c9aeb, State: Initialized, Role: FOLLOWER
I20240426 13:15:23.222390  8178 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5272f02dc6d4409f8d4754df958c9aeb" member_type: VOTER }
I20240426 13:15:23.222463  8178 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:15:23.222508  8178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:15:23.222561  8178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:23.223237  8178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5272f02dc6d4409f8d4754df958c9aeb" member_type: VOTER }
I20240426 13:15:23.223373  8178 leader_election.cc:302] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5272f02dc6d4409f8d4754df958c9aeb; no voters: 
I20240426 13:15:23.223551  8178 leader_election.cc:288] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:15:23.223637  8181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:15:23.223820  8181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 1 LEADER]: Becoming Leader. State: Replica: 5272f02dc6d4409f8d4754df958c9aeb, State: Running, Role: LEADER
I20240426 13:15:23.223935  8178 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:15:23.223949  8181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5272f02dc6d4409f8d4754df958c9aeb" member_type: VOTER }
I20240426 13:15:23.224397  8182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5272f02dc6d4409f8d4754df958c9aeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5272f02dc6d4409f8d4754df958c9aeb" member_type: VOTER } }
I20240426 13:15:23.224404  8183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5272f02dc6d4409f8d4754df958c9aeb. Latest consensus state: current_term: 1 leader_uuid: "5272f02dc6d4409f8d4754df958c9aeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5272f02dc6d4409f8d4754df958c9aeb" member_type: VOTER } }
I20240426 13:15:23.224684  8183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [sys.catalog]: This master's current role is: LEADER
I20240426 13:15:23.224578  8182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [sys.catalog]: This master's current role is: LEADER
I20240426 13:15:23.225160  8190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:15:23.225795  8190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:15:23.225976   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:15:23.227219  8190 catalog_manager.cc:1347] Generated new cluster ID: 88283b0616294d389da050fe840dbad7
I20240426 13:15:23.227264  8190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:15:23.258460  8190 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:15:23.258857  8190 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:15:23.264669  8190 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb: Generated new TSK 0
I20240426 13:15:23.264775  8190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:15:23.290215   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:23.291788  8199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:23.292644  8200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:23.293503  8202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:23.294414   398 server_base.cc:1034] running on GCE node
I20240426 13:15:23.294541   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:23.294574   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:23.294593   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137323294592 us; error 0 us; skew 500 ppm
I20240426 13:15:23.295325   398 webserver.cc:460] Webserver started at http://127.0.99.129:41935/ using document root <none> and password file <none>
I20240426 13:15:23.295459   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:23.295502   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:23.295565   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:23.295881   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "f74536e3f4b14ae0bca92b3048952ef7"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:23.297147   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:23.297978  8207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.298192   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:23.298245   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "f74536e3f4b14ae0bca92b3048952ef7"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:23.298288   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:23.329339   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:23.329550   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:23.329720   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:23.330087   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:23.330119   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.330143   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:23.330170   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.333495   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:33841
I20240426 13:15:23.333549  8270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:33841 every 8 connection(s)
I20240426 13:15:23.334221   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:23.335916  8276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:23.336709  8278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:23.337293  8275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:23.338093   398 server_base.cc:1034] running on GCE node
I20240426 13:15:23.338253  8271 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37749
I20240426 13:15:23.338274   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:23.338308   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:23.338327   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137323338328 us; error 0 us; skew 500 ppm
I20240426 13:15:23.338339  8271 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:23.338518  8271 heartbeater.cc:511] Master 127.0.99.190:37749 requested a full tablet report, sending...
I20240426 13:15:23.339068  8142 ts_manager.cc:194] Registered new tserver with Master: f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841)
I20240426 13:15:23.339699  8142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55270
I20240426 13:15:23.340261   398 webserver.cc:460] Webserver started at http://127.0.99.130:40767/ using document root <none> and password file <none>
I20240426 13:15:23.340384   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:23.340430   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:23.340503   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:23.340806   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "4a3fa56fbd164853a753bf63b900bb68"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:23.342092   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:15:23.342819  8283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.343006   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:15:23.343080   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "4a3fa56fbd164853a753bf63b900bb68"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:23.343142   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:23.356261   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:23.356508   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:23.356726   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:23.357082   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:23.357118   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.357161   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:23.357193   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.360813   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:41631
I20240426 13:15:23.360834  8346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:41631 every 8 connection(s)
I20240426 13:15:23.361500   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:23.363035  8350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:23.363567  8351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:23.364485  8353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:23.365104  8347 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37749
I20240426 13:15:23.365185  8347 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:23.365335   398 server_base.cc:1034] running on GCE node
I20240426 13:15:23.365386  8347 heartbeater.cc:511] Master 127.0.99.190:37749 requested a full tablet report, sending...
I20240426 13:15:23.365507   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:23.365548   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:23.365571   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137323365571 us; error 0 us; skew 500 ppm
I20240426 13:15:23.365938  8142 ts_manager.cc:194] Registered new tserver with Master: 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631)
I20240426 13:15:23.366451   398 webserver.cc:460] Webserver started at http://127.0.99.131:44551/ using document root <none> and password file <none>
I20240426 13:15:23.366580   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:23.366616  8142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55286
I20240426 13:15:23.366643   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:23.366730   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:23.367069   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "97b787ee0ff24a7d978bcef36670574e"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:23.368360   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:23.369127  8358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.369339   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:23.369400   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "97b787ee0ff24a7d978bcef36670574e"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:23.369463   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:23.374389   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:23.374667   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:23.374895   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:23.375232   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:23.375268   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.375326   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:23.375360   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:23.379433   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:45705
I20240426 13:15:23.379480  8421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:45705 every 8 connection(s)
I20240426 13:15:23.382797  8422 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37749
I20240426 13:15:23.382861  8422 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:23.383008  8422 heartbeater.cc:511] Master 127.0.99.190:37749 requested a full tablet report, sending...
I20240426 13:15:23.383524  8142 ts_manager.cc:194] Registered new tserver with Master: 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:23.383947  8142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55294
I20240426 13:15:23.384218   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004469013s
I20240426 13:15:23.384646   398 test_util.cc:268] Using random seed: -1304318138
I20240426 13:15:23.390384  8142 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55296:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:15:23.390713  8142 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:15:23.398145  8235 tablet_service.cc:1467] Processing CreateTablet for tablet 35f8baefed1c436fa9b04a33f36f209c (DEFAULT_TABLE table=test-workload [id=af8fba02c72c407aaa12cf0f498031ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:23.398406  8235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35f8baefed1c436fa9b04a33f36f209c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:23.399236  8311 tablet_service.cc:1467] Processing CreateTablet for tablet 35f8baefed1c436fa9b04a33f36f209c (DEFAULT_TABLE table=test-workload [id=af8fba02c72c407aaa12cf0f498031ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:23.399457  8311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35f8baefed1c436fa9b04a33f36f209c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:23.399662  8386 tablet_service.cc:1467] Processing CreateTablet for tablet 35f8baefed1c436fa9b04a33f36f209c (DEFAULT_TABLE table=test-workload [id=af8fba02c72c407aaa12cf0f498031ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:23.399835  8386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35f8baefed1c436fa9b04a33f36f209c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:23.400537  8437 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Bootstrap starting.
I20240426 13:15:23.401404  8437 tablet_bootstrap.cc:654] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:23.401641  8439 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Bootstrap starting.
I20240426 13:15:23.401933  8438 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Bootstrap starting.
I20240426 13:15:23.402356  8439 tablet_bootstrap.cc:654] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:23.402420  8437 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: No bootstrap required, opened a new log
I20240426 13:15:23.402493  8437 ts_tablet_manager.cc:1367] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:15:23.402704  8438 tablet_bootstrap.cc:654] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:23.402882  8437 raft_consensus.cc:348] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.402998  8437 raft_consensus.cc:374] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:23.403045  8437 raft_consensus.cc:729] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74536e3f4b14ae0bca92b3048952ef7, State: Initialized, Role: FOLLOWER
I20240426 13:15:23.403177  8437 consensus_queue.cc:260] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.403460  8437 ts_tablet_manager.cc:1398] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:23.403528  8271 heartbeater.cc:503] Master 127.0.99.190:37749 was elected leader, sending a full tablet report...
I20240426 13:15:23.404081  8439 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: No bootstrap required, opened a new log
I20240426 13:15:23.404224  8439 ts_tablet_manager.cc:1367] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Time spent bootstrapping tablet: real 0.003s	user 0.000s	sys 0.002s
I20240426 13:15:23.404253  8438 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: No bootstrap required, opened a new log
I20240426 13:15:23.404428  8438 ts_tablet_manager.cc:1367] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Time spent bootstrapping tablet: real 0.003s	user 0.000s	sys 0.002s
I20240426 13:15:23.404644  8439 raft_consensus.cc:348] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.404726  8439 raft_consensus.cc:374] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:23.404757  8439 raft_consensus.cc:729] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b787ee0ff24a7d978bcef36670574e, State: Initialized, Role: FOLLOWER
I20240426 13:15:23.404793  8438 raft_consensus.cc:348] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.404901  8439 consensus_queue.cc:260] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.404940  8438 raft_consensus.cc:374] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:23.404999  8438 raft_consensus.cc:729] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a3fa56fbd164853a753bf63b900bb68, State: Initialized, Role: FOLLOWER
I20240426 13:15:23.405150  8438 consensus_queue.cc:260] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.405217  8439 ts_tablet_manager.cc:1398] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:15:23.405357  8422 heartbeater.cc:503] Master 127.0.99.190:37749 was elected leader, sending a full tablet report...
I20240426 13:15:23.405465  8438 ts_tablet_manager.cc:1398] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:15:23.405649  8347 heartbeater.cc:503] Master 127.0.99.190:37749 was elected leader, sending a full tablet report...
W20240426 13:15:23.584156  8272 tablet.cc:2369] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 13:15:23.645699  8443 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:15:23.645823  8443 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.646323  8443 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631), 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:23.649859  8396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b787ee0ff24a7d978bcef36670574e" is_pre_election: true
I20240426 13:15:23.649859  8321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a3fa56fbd164853a753bf63b900bb68" is_pre_election: true
I20240426 13:15:23.650029  8396 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f74536e3f4b14ae0bca92b3048952ef7 in term 0.
I20240426 13:15:23.650040  8321 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f74536e3f4b14ae0bca92b3048952ef7 in term 0.
I20240426 13:15:23.650363  8208 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a3fa56fbd164853a753bf63b900bb68, f74536e3f4b14ae0bca92b3048952ef7; no voters: 
I20240426 13:15:23.650514  8443 raft_consensus.cc:2781] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:15:23.650568  8443 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:15:23.650625  8443 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:23.651332  8443 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.651589  8443 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 1 election: Requested vote from peers 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631), 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:23.651824  8321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a3fa56fbd164853a753bf63b900bb68"
I20240426 13:15:23.651840  8396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97b787ee0ff24a7d978bcef36670574e"
I20240426 13:15:23.651919  8321 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:23.651928  8396 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:23.652707  8396 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f74536e3f4b14ae0bca92b3048952ef7 in term 1.
I20240426 13:15:23.652731  8321 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f74536e3f4b14ae0bca92b3048952ef7 in term 1.
I20240426 13:15:23.652940  8209 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b787ee0ff24a7d978bcef36670574e, f74536e3f4b14ae0bca92b3048952ef7; no voters: 
I20240426 13:15:23.653120  8443 raft_consensus.cc:2781] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:15:23.653340  8443 raft_consensus.cc:686] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 1 LEADER]: Becoming Leader. State: Replica: f74536e3f4b14ae0bca92b3048952ef7, State: Running, Role: LEADER
I20240426 13:15:23.653501  8443 consensus_queue.cc:237] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:23.655030  8141 catalog_manager.cc:5484] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "f74536e3f4b14ae0bca92b3048952ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } health_report { overall_health: UNKNOWN } } }
I20240426 13:15:23.698338  8321 raft_consensus.cc:1261] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 1 FOLLOWER]: Refusing update from remote peer f74536e3f4b14ae0bca92b3048952ef7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:15:23.698343  8396 raft_consensus.cc:1261] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 1 FOLLOWER]: Refusing update from remote peer f74536e3f4b14ae0bca92b3048952ef7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:15:23.698683  8447 consensus_queue.cc:1035] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:15:23.698905  8443 consensus_queue.cc:1035] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:15:23.708022  8455 mvcc.cc:205] Tried to move back new op lower bound from 7021106477868888064 to 7021106477686001664. Current Snapshot: MvccSnapshot[applied={T|T < 7021106477865680896}]
I20240426 13:15:33.707808   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:15:33.710954   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:33.711112   398 tablet_replica.cc:330] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: stopping tablet replica
I20240426 13:15:33.711277   398 raft_consensus.cc:2229] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:15:33.711409   398 raft_consensus.cc:2258] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:33.717490   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:15:33.719022   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:33.719874  8498 meta_cache.cc:302] tablet 35f8baefed1c436fa9b04a33f36f209c: replica f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841) has failed: Network error: Client connection negotiation failed: client connection to 127.0.99.129:33841: connect: Connection refused (error 111)
W20240426 13:15:33.721346  8507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:33.721540  8504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:33.722401  8505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:33.723191   398 server_base.cc:1034] running on GCE node
I20240426 13:15:33.723315   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:33.723357   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:33.723373   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137333723374 us; error 0 us; skew 500 ppm
I20240426 13:15:33.724347   398 webserver.cc:460] Webserver started at http://127.0.99.129:41935/ using document root <none> and password file <none>
I20240426 13:15:33.724504   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:33.724542   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:15:33.725643   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:33.726433  8513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:33.726644   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:33.726699   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "f74536e3f4b14ae0bca92b3048952ef7"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:33.726759   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20240426 13:15:33.728821  8311 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48036: Illegal state: replica 4a3fa56fbd164853a753bf63b900bb68 is not leader of this config: current role FOLLOWER
I20240426 13:15:33.730320   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:33.730532   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:33.730721   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:33.731142  8520 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240426 13:15:33.731940   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:15:33.731977   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:33.732023   398 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240426 13:15:33.732566   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:15:33.732600   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:33.732647  8520 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Bootstrap starting.
I20240426 13:15:33.736886   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:33841
I20240426 13:15:33.738269  8579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:33841 every 8 connection(s)
W20240426 13:15:33.741087  8386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:42220: Illegal state: replica 97b787ee0ff24a7d978bcef36670574e is not leader of this config: current role FOLLOWER
I20240426 13:15:33.742786  8580 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37749
I20240426 13:15:33.742878  8580 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:33.743043  8580 heartbeater.cc:511] Master 127.0.99.190:37749 requested a full tablet report, sending...
I20240426 13:15:33.743693  8133 ts_manager.cc:194] Re-registered known tserver with Master: f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841)
I20240426 13:15:33.744855  8133 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59118
I20240426 13:15:34.745697  8580 heartbeater.cc:503] Master 127.0.99.190:37749 was elected leader, sending a full tablet report...
I20240426 13:15:35.190567  8584 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f74536e3f4b14ae0bca92b3048952ef7)
I20240426 13:15:35.190740  8584 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:35.191579  8584 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841), 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:35.192407  8589 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f74536e3f4b14ae0bca92b3048952ef7)
I20240426 13:15:35.192557  8589 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:35.193344  8589 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841), 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631)
I20240426 13:15:35.196240  8554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "4a3fa56fbd164853a753bf63b900bb68" candidate_term: 2 candidate_status { last_received { term: 1 index: 3115 } } ignore_live_leader: false dest_uuid: "f74536e3f4b14ae0bca92b3048952ef7" is_pre_election: true
I20240426 13:15:35.196281  8394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "4a3fa56fbd164853a753bf63b900bb68" candidate_term: 2 candidate_status { last_received { term: 1 index: 3115 } } ignore_live_leader: false dest_uuid: "97b787ee0ff24a7d978bcef36670574e" is_pre_election: true
I20240426 13:15:35.196352  8394 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a3fa56fbd164853a753bf63b900bb68 in term 1.
I20240426 13:15:35.196611  8285 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a3fa56fbd164853a753bf63b900bb68, 97b787ee0ff24a7d978bcef36670574e; no voters: 
W20240426 13:15:35.196694  8286 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240426 13:15:35.196940  8584 raft_consensus.cc:2781] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240426 13:15:35.197007  8584 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f74536e3f4b14ae0bca92b3048952ef7)
I20240426 13:15:35.197098  8584 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:35.198194  8584 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:35.198552  8584 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 2 election: Requested vote from peers f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841), 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:35.198894  8554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "4a3fa56fbd164853a753bf63b900bb68" candidate_term: 2 candidate_status { last_received { term: 1 index: 3115 } } ignore_live_leader: false dest_uuid: "f74536e3f4b14ae0bca92b3048952ef7"
W20240426 13:15:35.199189  8286 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240426 13:15:35.199298  8554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "97b787ee0ff24a7d978bcef36670574e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3115 } } ignore_live_leader: false dest_uuid: "f74536e3f4b14ae0bca92b3048952ef7" is_pre_election: true
I20240426 13:15:35.199393  8321 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "97b787ee0ff24a7d978bcef36670574e" candidate_term: 2 candidate_status { last_received { term: 1 index: 3115 } } ignore_live_leader: false dest_uuid: "4a3fa56fbd164853a753bf63b900bb68" is_pre_election: true
I20240426 13:15:35.199550  8394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "4a3fa56fbd164853a753bf63b900bb68" candidate_term: 2 candidate_status { last_received { term: 1 index: 3115 } } ignore_live_leader: false dest_uuid: "97b787ee0ff24a7d978bcef36670574e"
W20240426 13:15:35.199621  8361 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841): Illegal state: must be running to vote when last-logged opid is not known
I20240426 13:15:35.199556  8321 raft_consensus.cc:2379] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 97b787ee0ff24a7d978bcef36670574e in current term 2: Already voted for candidate 4a3fa56fbd164853a753bf63b900bb68 in this term.
I20240426 13:15:35.199649  8394 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:35.199971  8359 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 97b787ee0ff24a7d978bcef36670574e; no voters: 4a3fa56fbd164853a753bf63b900bb68, f74536e3f4b14ae0bca92b3048952ef7
I20240426 13:15:35.200548  8394 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a3fa56fbd164853a753bf63b900bb68 in term 2.
I20240426 13:15:35.200660  8589 raft_consensus.cc:2726] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20240426 13:15:35.200783  8285 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4a3fa56fbd164853a753bf63b900bb68, 97b787ee0ff24a7d978bcef36670574e; no voters: f74536e3f4b14ae0bca92b3048952ef7
I20240426 13:15:35.200929  8584 raft_consensus.cc:2781] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:15:35.201211  8584 raft_consensus.cc:686] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 LEADER]: Becoming Leader. State: Replica: 4a3fa56fbd164853a753bf63b900bb68, State: Running, Role: LEADER
I20240426 13:15:35.201385  8584 consensus_queue.cc:237] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3115, Committed index: 3115, Last appended: 1.3115, Last appended by leader: 3115, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:35.202689  8133 catalog_manager.cc:5484] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 reported cstate change: term changed from 1 to 2, leader changed from f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129) to 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130). New cstate: current_term: 2 leader_uuid: "4a3fa56fbd164853a753bf63b900bb68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } health_report { overall_health: UNKNOWN } } }
W20240426 13:15:35.626688  8286 consensus_peers.cc:485] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 -> Peer f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841): Couldn't send request to peer f74536e3f4b14ae0bca92b3048952ef7. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20240426 13:15:35.760344  8394 raft_consensus.cc:1261] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 2 FOLLOWER]: Refusing update from remote peer 4a3fa56fbd164853a753bf63b900bb68: Log matching property violated. Preceding OpId in replica: term: 1 index: 3115. Preceding OpId from leader: term: 2 index: 3116. (index mismatch)
I20240426 13:15:35.760731  8584 consensus_queue.cc:1035] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3116, Last known committed idx: 3115, Time since last communication: 0.000s
I20240426 13:15:37.888743  8520 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Bootstrap replayed 1/1 log segments. Stats: ops{read=3115 overwritten=0 applied=3115 ignored=0} inserts{seen=155700 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:15:37.889052  8520 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Bootstrap complete.
I20240426 13:15:37.890121  8520 ts_tablet_manager.cc:1367] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Time spent bootstrapping tablet: real 4.158s	user 4.032s	sys 0.124s
I20240426 13:15:37.890440  8520 raft_consensus.cc:348] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:37.890532  8520 raft_consensus.cc:729] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f74536e3f4b14ae0bca92b3048952ef7, State: Initialized, Role: FOLLOWER
I20240426 13:15:37.890614  8520 consensus_queue.cc:260] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3115, Last appended: 1.3115, Last appended by leader: 3115, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:37.890846  8520 ts_tablet_manager.cc:1398] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
W20240426 13:15:37.978025  8537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45302: Illegal state: replica f74536e3f4b14ae0bca92b3048952ef7 is not leader of this config: current role FOLLOWER
I20240426 13:15:38.352731  8554 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:15:38.391121   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:15:38.395409   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:38.395656   398 tablet_replica.cc:330] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: stopping tablet replica
I20240426 13:15:38.395774   398 raft_consensus.cc:2229] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:15:38.395929   398 raft_consensus.cc:2258] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 FOLLOWER]: Raft consensus is shut down!
W20240426 13:15:38.400739   398 mvcc.cc:119] aborting op with timestamp 7021106538049183744 in state 0; MVCC is closed
W20240426 13:15:38.408025  8496 meta_cache.cc:302] tablet 35f8baefed1c436fa9b04a33f36f209c: replica 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631) has failed: Network error: Client connection negotiation failed: client connection to 127.0.99.130:41631: connect: Connection refused (error 111)
I20240426 13:15:38.413262   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
W20240426 13:15:38.414759  8537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45302: Illegal state: replica f74536e3f4b14ae0bca92b3048952ef7 is not leader of this config: current role FOLLOWER
I20240426 13:15:38.415094   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:38.416805  8621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:38.417805  8623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:38.418566  8620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:38.419407   398 server_base.cc:1034] running on GCE node
I20240426 13:15:38.419610   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:38.419651   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:38.419677   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137338419675 us; error 0 us; skew 500 ppm
I20240426 13:15:38.420619   398 webserver.cc:460] Webserver started at http://127.0.99.130:40767/ using document root <none> and password file <none>
I20240426 13:15:38.420758   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:38.420799   398 fs_manager.cc:362] Using existing metadata directory in first data directory
W20240426 13:15:38.421743  8386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:42220: Illegal state: replica 97b787ee0ff24a7d978bcef36670574e is not leader of this config: current role FOLLOWER
I20240426 13:15:38.421996   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:38.422683  8628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:38.422894   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:15:38.422951   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "4a3fa56fbd164853a753bf63b900bb68"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:38.423017   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:38.433215   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:38.433435   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:38.433637   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:38.434119  8635 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240426 13:15:38.434989   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:15:38.435032   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:38.435082   398 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240426 13:15:38.435655   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:15:38.435693   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:38.435731  8635 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Bootstrap starting.
I20240426 13:15:38.440099   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:41631
W20240426 13:15:38.442018  8537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45302: Illegal state: replica f74536e3f4b14ae0bca92b3048952ef7 is not leader of this config: current role FOLLOWER
I20240426 13:15:38.442057  8693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:41631 every 8 connection(s)
I20240426 13:15:38.449335  8694 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37749
I20240426 13:15:38.449407  8694 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:38.449568  8694 heartbeater.cc:511] Master 127.0.99.190:37749 requested a full tablet report, sending...
I20240426 13:15:38.450186  8142 ts_manager.cc:194] Re-registered known tserver with Master: 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631)
I20240426 13:15:38.450970  8142 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59132
W20240426 13:15:38.452824  8386 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:42220: Illegal state: replica 97b787ee0ff24a7d978bcef36670574e is not leader of this config: current role FOLLOWER
I20240426 13:15:39.451699  8694 heartbeater.cc:503] Master 127.0.99.190:37749 was elected leader, sending a full tablet report...
I20240426 13:15:39.893741  8699 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4a3fa56fbd164853a753bf63b900bb68)
I20240426 13:15:39.893863  8699 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:39.894398  8700 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4a3fa56fbd164853a753bf63b900bb68)
I20240426 13:15:39.894577  8699 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631), 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:39.894562  8700 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:39.895047  8700 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841), 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631)
I20240426 13:15:39.895289  8554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "97b787ee0ff24a7d978bcef36670574e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3259 } } ignore_live_leader: false dest_uuid: "f74536e3f4b14ae0bca92b3048952ef7" is_pre_election: true
I20240426 13:15:39.895401  8554 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97b787ee0ff24a7d978bcef36670574e in term 2.
I20240426 13:15:39.895677  8361 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b787ee0ff24a7d978bcef36670574e, f74536e3f4b14ae0bca92b3048952ef7; no voters: 
I20240426 13:15:39.895896  8700 raft_consensus.cc:2781] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20240426 13:15:39.895987  8700 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4a3fa56fbd164853a753bf63b900bb68)
I20240426 13:15:39.896106  8700 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:15:39.897214  8700 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:39.897810  8554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "97b787ee0ff24a7d978bcef36670574e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3259 } } ignore_live_leader: false dest_uuid: "f74536e3f4b14ae0bca92b3048952ef7"
I20240426 13:15:39.897902  8554 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:15:39.898859  8554 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b787ee0ff24a7d978bcef36670574e in term 3.
I20240426 13:15:39.899150  8361 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97b787ee0ff24a7d978bcef36670574e, f74536e3f4b14ae0bca92b3048952ef7; no voters: 
I20240426 13:15:39.899363  8668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3259 } } ignore_live_leader: false dest_uuid: "4a3fa56fbd164853a753bf63b900bb68" is_pre_election: true
I20240426 13:15:39.899459  8700 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 3 election: Requested vote from peers f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841), 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631)
I20240426 13:15:39.899546  8700 raft_consensus.cc:2781] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 3 FOLLOWER]: Leader election won for term 3
I20240426 13:15:39.899592  8700 raft_consensus.cc:686] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 3 LEADER]: Becoming Leader. State: Replica: 97b787ee0ff24a7d978bcef36670574e, State: Running, Role: LEADER
I20240426 13:15:39.899704  8700 consensus_queue.cc:237] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3259, Committed index: 3259, Last appended: 2.3259, Last appended by leader: 3259, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
W20240426 13:15:39.899726  8514 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631): Illegal state: must be running to vote when last-logged opid is not known
I20240426 13:15:39.899986  8667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "97b787ee0ff24a7d978bcef36670574e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3259 } } ignore_live_leader: false dest_uuid: "4a3fa56fbd164853a753bf63b900bb68"
I20240426 13:15:39.900303  8668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "97b787ee0ff24a7d978bcef36670574e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3259 } } ignore_live_leader: false dest_uuid: "4a3fa56fbd164853a753bf63b900bb68" is_pre_election: true
W20240426 13:15:39.900677  8359 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631): Illegal state: must be running to vote when last-logged opid is not known
W20240426 13:15:39.900818  8359 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631): Illegal state: must be running to vote when last-logged opid is not known
I20240426 13:15:39.901091  8142 catalog_manager.cc:5484] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e reported cstate change: term changed from 2 to 3, leader changed from 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130) to 97b787ee0ff24a7d978bcef36670574e (127.0.99.131). New cstate: current_term: 3 leader_uuid: "97b787ee0ff24a7d978bcef36670574e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } health_report { overall_health: HEALTHY } } }
I20240426 13:15:39.902290  8394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3259 } } ignore_live_leader: false dest_uuid: "97b787ee0ff24a7d978bcef36670574e" is_pre_election: true
I20240426 13:15:39.902590  8515 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f74536e3f4b14ae0bca92b3048952ef7; no voters: 4a3fa56fbd164853a753bf63b900bb68, 97b787ee0ff24a7d978bcef36670574e
I20240426 13:15:39.902792  8699 raft_consensus.cc:2726] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20240426 13:15:40.450471  8359 consensus_peers.cc:485] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e -> Peer 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631): Couldn't send request to peer 4a3fa56fbd164853a753bf63b900bb68. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20240426 13:15:40.485534  8554 raft_consensus.cc:1261] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 3 FOLLOWER]: Refusing update from remote peer 97b787ee0ff24a7d978bcef36670574e: Log matching property violated. Preceding OpId in replica: term: 2 index: 3259. Preceding OpId from leader: term: 3 index: 3260. (index mismatch)
I20240426 13:15:40.485940  8713 consensus_queue.cc:1035] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3260, Last known committed idx: 3259, Time since last communication: 0.000s
W20240426 13:15:42.615290  8359 consensus_peers.cc:485] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e -> Peer 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631): Couldn't send request to peer 4a3fa56fbd164853a753bf63b900bb68. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20240426 13:15:42.643326  8635 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Bootstrap replayed 1/1 log segments. Stats: ops{read=3259 overwritten=0 applied=3259 ignored=0} inserts{seen=155700 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:15:42.643662  8635 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Bootstrap complete.
I20240426 13:15:42.644697  8635 ts_tablet_manager.cc:1367] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Time spent bootstrapping tablet: real 4.209s	user 4.115s	sys 0.088s
I20240426 13:15:42.645041  8635 raft_consensus.cc:348] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:42.645160  8635 raft_consensus.cc:729] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a3fa56fbd164853a753bf63b900bb68, State: Initialized, Role: FOLLOWER
I20240426 13:15:42.645299  8635 consensus_queue.cc:260] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3259, Last appended: 2.3259, Last appended by leader: 3259, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:42.645567  8635 ts_tablet_manager.cc:1398] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68: Time spent starting tablet: real 0.001s	user 0.003s	sys 0.000s
W20240426 13:15:42.655452  8658 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48078: Illegal state: replica 4a3fa56fbd164853a753bf63b900bb68 is not leader of this config: current role FOLLOWER
W20240426 13:15:42.750501  8537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45302: Illegal state: replica f74536e3f4b14ae0bca92b3048952ef7 is not leader of this config: current role FOLLOWER
I20240426 13:15:42.995831  8668 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:15:43.145876   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:15:43.150188   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:43.150321   398 tablet_replica.cc:330] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: stopping tablet replica
I20240426 13:15:43.150418   398 raft_consensus.cc:2229] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 3 LEADER]: Raft consensus shutting down.
I20240426 13:15:43.150535   398 raft_consensus.cc:2258] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 3 FOLLOWER]: Raft consensus is shut down!
W20240426 13:15:43.158594  8497 meta_cache.cc:302] tablet 35f8baefed1c436fa9b04a33f36f209c: replica 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705) has failed: Network error: Client connection negotiation failed: client connection to 127.0.99.131:45705: connect: Connection refused (error 111)
W20240426 13:15:43.163213  8537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45302: Illegal state: replica f74536e3f4b14ae0bca92b3048952ef7 is not leader of this config: current role FOLLOWER
I20240426 13:15:43.167114   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:15:43.168804   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:43.169013  8658 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48078: Illegal state: replica 4a3fa56fbd164853a753bf63b900bb68 is not leader of this config: current role FOLLOWER
W20240426 13:15:43.170562  8735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:43.171430  8738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:43.172284  8736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:43.172982   398 server_base.cc:1034] running on GCE node
I20240426 13:15:43.173120   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:43.173156   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:43.173174   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137343173174 us; error 0 us; skew 500 ppm
I20240426 13:15:43.178812   398 webserver.cc:460] Webserver started at http://127.0.99.131:44551/ using document root <none> and password file <none>
I20240426 13:15:43.178948   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:43.178983   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:15:43.180001   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:43.180727  8743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:43.180932   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:43.180991   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "97b787ee0ff24a7d978bcef36670574e"
format_stamp: "Formatted at 2024-04-26 13:15:23 on dist-test-slave-2jsl"
I20240426 13:15:43.181058   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20240426 13:15:43.189352  8537 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:45302: Illegal state: replica f74536e3f4b14ae0bca92b3048952ef7 is not leader of this config: current role FOLLOWER
W20240426 13:15:43.197168  8658 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:48078: Illegal state: replica 4a3fa56fbd164853a753bf63b900bb68 is not leader of this config: current role FOLLOWER
I20240426 13:15:43.199981   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:43.200206   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:43.200402   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:43.200892  8750 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240426 13:15:43.201751   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:15:43.201790   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:43.201817   398 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240426 13:15:43.202437   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:15:43.202471   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:43.202520  8750 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Bootstrap starting.
I20240426 13:15:43.206964   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:45705
I20240426 13:15:43.209036  8808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:45705 every 8 connection(s)
I20240426 13:15:43.213277  8809 heartbeater.cc:347] Connected to a master server at 127.0.99.190:37749
I20240426 13:15:43.213356  8809 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:43.213513  8809 heartbeater.cc:511] Master 127.0.99.190:37749 requested a full tablet report, sending...
I20240426 13:15:43.214118  8133 ts_manager.cc:194] Re-registered known tserver with Master: 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:43.214745  8133 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57970
I20240426 13:15:44.215504  8809 heartbeater.cc:503] Master 127.0.99.190:37749 was elected leader, sending a full tablet report...
I20240426 13:15:44.648757  8814 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 97b787ee0ff24a7d978bcef36670574e)
I20240426 13:15:44.648869  8814 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:44.649223  8814 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631), 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:44.649561  8815 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 97b787ee0ff24a7d978bcef36670574e)
I20240426 13:15:44.649569  8668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3366 } } ignore_live_leader: false dest_uuid: "4a3fa56fbd164853a753bf63b900bb68" is_pre_election: true
I20240426 13:15:44.649694  8815 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:44.649986  8668 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f74536e3f4b14ae0bca92b3048952ef7 in term 3.
I20240426 13:15:44.650080  8815 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129:33841), 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:44.650511  8514 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a3fa56fbd164853a753bf63b900bb68, f74536e3f4b14ae0bca92b3048952ef7; no voters: 
I20240426 13:15:44.650697  8814 raft_consensus.cc:2781] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20240426 13:15:44.650744  8814 raft_consensus.cc:482] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 97b787ee0ff24a7d978bcef36670574e)
I20240426 13:15:44.650786  8814 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 3 FOLLOWER]: Advancing to term 4
I20240426 13:15:44.651840  8814 raft_consensus.cc:504] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:44.652137  8814 leader_election.cc:288] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 4 election: Requested vote from peers 4a3fa56fbd164853a753bf63b900bb68 (127.0.99.130:41631), 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705)
I20240426 13:15:44.652391  8668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3366 } } ignore_live_leader: false dest_uuid: "4a3fa56fbd164853a753bf63b900bb68"
I20240426 13:15:44.652487  8668 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 3 FOLLOWER]: Advancing to term 4
I20240426 13:15:44.653383  8668 raft_consensus.cc:2454] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f74536e3f4b14ae0bca92b3048952ef7 in term 4.
I20240426 13:15:44.653717  8514 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a3fa56fbd164853a753bf63b900bb68, f74536e3f4b14ae0bca92b3048952ef7; no voters: 
I20240426 13:15:44.653857  8814 raft_consensus.cc:2781] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 4 FOLLOWER]: Leader election won for term 4
I20240426 13:15:44.654073  8814 raft_consensus.cc:686] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 4 LEADER]: Becoming Leader. State: Replica: f74536e3f4b14ae0bca92b3048952ef7, State: Running, Role: LEADER
I20240426 13:15:44.654247  8814 consensus_queue.cc:237] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3366, Committed index: 3366, Last appended: 3.3366, Last appended by leader: 3366, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:44.654460  8783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3366 } } ignore_live_leader: false dest_uuid: "97b787ee0ff24a7d978bcef36670574e" is_pre_election: true
I20240426 13:15:44.654755  8782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "f74536e3f4b14ae0bca92b3048952ef7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3366 } } ignore_live_leader: false dest_uuid: "97b787ee0ff24a7d978bcef36670574e"
W20240426 13:15:44.655113  8515 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 4 pre-election: Tablet error from VoteRequest() call to peer 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705): Illegal state: must be running to vote when last-logged opid is not known
W20240426 13:15:44.655270  8515 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [CANDIDATE]: Term 4 election: Tablet error from VoteRequest() call to peer 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705): Illegal state: must be running to vote when last-logged opid is not known
I20240426 13:15:44.656002  8133 catalog_manager.cc:5484] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 reported cstate change: term changed from 3 to 4, leader changed from 97b787ee0ff24a7d978bcef36670574e (127.0.99.131) to f74536e3f4b14ae0bca92b3048952ef7 (127.0.99.129). New cstate: current_term: 4 leader_uuid: "f74536e3f4b14ae0bca92b3048952ef7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } health_report { overall_health: UNKNOWN } } }
I20240426 13:15:44.657162  8554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "4a3fa56fbd164853a753bf63b900bb68" candidate_term: 4 candidate_status { last_received { term: 3 index: 3366 } } ignore_live_leader: false dest_uuid: "f74536e3f4b14ae0bca92b3048952ef7" is_pre_election: true
I20240426 13:15:44.657289  8782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35f8baefed1c436fa9b04a33f36f209c" candidate_uuid: "4a3fa56fbd164853a753bf63b900bb68" candidate_term: 4 candidate_status { last_received { term: 3 index: 3366 } } ignore_live_leader: false dest_uuid: "97b787ee0ff24a7d978bcef36670574e" is_pre_election: true
W20240426 13:15:44.657588  8630 leader_election.cc:341] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 4 pre-election: Tablet error from VoteRequest() call to peer 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705): Illegal state: must be running to vote when last-logged opid is not known
I20240426 13:15:44.657652  8630 leader_election.cc:302] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4a3fa56fbd164853a753bf63b900bb68; no voters: 97b787ee0ff24a7d978bcef36670574e, f74536e3f4b14ae0bca92b3048952ef7
I20240426 13:15:44.657765  8815 raft_consensus.cc:2726] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20240426 13:15:45.087102  8668 raft_consensus.cc:1261] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 4 FOLLOWER]: Refusing update from remote peer f74536e3f4b14ae0bca92b3048952ef7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3366. Preceding OpId from leader: term: 4 index: 3367. (index mismatch)
I20240426 13:15:45.087672  8822 consensus_queue.cc:1035] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3367, Last known committed idx: 3366, Time since last communication: 0.000s
W20240426 13:15:45.090708  8515 consensus_peers.cc:485] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 -> Peer 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705): Couldn't send request to peer 97b787ee0ff24a7d978bcef36670574e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20240426 13:15:47.124874  8515 consensus_peers.cc:485] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 -> Peer 97b787ee0ff24a7d978bcef36670574e (127.0.99.131:45705): Couldn't send request to peer 97b787ee0ff24a7d978bcef36670574e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20240426 13:15:47.511305  8750 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3366 overwritten=0 applied=3366 ignored=0} inserts{seen=155700 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:15:47.511638  8750 tablet_bootstrap.cc:492] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Bootstrap complete.
I20240426 13:15:47.512619  8750 ts_tablet_manager.cc:1367] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Time spent bootstrapping tablet: real 4.310s	user 4.184s	sys 0.120s
I20240426 13:15:47.512944  8750 raft_consensus.cc:348] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:47.513037  8750 raft_consensus.cc:729] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b787ee0ff24a7d978bcef36670574e, State: Initialized, Role: FOLLOWER
I20240426 13:15:47.513121  8750 consensus_queue.cc:260] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3366, Last appended: 3.3366, Last appended by leader: 3366, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f74536e3f4b14ae0bca92b3048952ef7" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 33841 } } peers { permanent_uuid: "4a3fa56fbd164853a753bf63b900bb68" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 41631 } } peers { permanent_uuid: "97b787ee0ff24a7d978bcef36670574e" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 45705 } }
I20240426 13:15:47.513357  8750 ts_tablet_manager.cc:1398] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e: Time spent starting tablet: real 0.001s	user 0.003s	sys 0.000s
W20240426 13:15:47.515211  8773 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:56306: Illegal state: replica 97b787ee0ff24a7d978bcef36670574e is not leader of this config: current role FOLLOWER
I20240426 13:15:47.633886  8782 raft_consensus.cc:3037] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 3 FOLLOWER]: Advancing to term 4
I20240426 13:15:48.017374   398 tablet_server.cc:178] TabletServer@127.0.99.129:33841 shutting down...
I20240426 13:15:48.020956   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:48.021086   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:15:48.021174   398 raft_consensus.cc:2229] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 4 LEADER]: Raft consensus shutting down.
I20240426 13:15:48.021330   398 raft_consensus.cc:2258] T 35f8baefed1c436fa9b04a33f36f209c P f74536e3f4b14ae0bca92b3048952ef7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:48.038528   398 tablet_server.cc:194] TabletServer@127.0.99.129:33841 shutdown complete.
I20240426 13:15:48.040732   398 tablet_server.cc:178] TabletServer@127.0.99.130:41631 shutting down...
I20240426 13:15:48.044437   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:48.044569   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:15:48.044673   398 raft_consensus.cc:2229] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 4 FOLLOWER]: Raft consensus shutting down.
I20240426 13:15:48.044751   398 raft_consensus.cc:2258] T 35f8baefed1c436fa9b04a33f36f209c P 4a3fa56fbd164853a753bf63b900bb68 [term 4 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:48.061656   398 tablet_server.cc:194] TabletServer@127.0.99.130:41631 shutdown complete.
I20240426 13:15:48.063457   398 tablet_server.cc:178] TabletServer@127.0.99.131:45705 shutting down...
I20240426 13:15:48.067399   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:15:48.067553   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:15:48.067689   398 raft_consensus.cc:2229] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 4 FOLLOWER]: Raft consensus shutting down.
I20240426 13:15:48.067780   398 raft_consensus.cc:2258] T 35f8baefed1c436fa9b04a33f36f209c P 97b787ee0ff24a7d978bcef36670574e [term 4 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:48.085485   398 tablet_server.cc:194] TabletServer@127.0.99.131:45705 shutdown complete.
I20240426 13:15:48.087607   398 master.cc:537] Master@127.0.99.190:37749 shutting down...
I20240426 13:15:48.091006   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:15:48.091112   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:48.091171   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5272f02dc6d4409f8d4754df958c9aeb: stopping tablet replica
I20240426 13:15:48.103788   398 master.cc:552] Master@127.0.99.190:37749 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientSucceedsOnBootstrap (24934 ms)
[ RUN      ] TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound
I20240426 13:15:48.112736   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:33783
I20240426 13:15:48.113062   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:48.115947  8849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:48.117075  8846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:48.117655  8847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:48.117935   398 server_base.cc:1034] running on GCE node
I20240426 13:15:48.118077   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:48.118114   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:48.118134   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137348118133 us; error 0 us; skew 500 ppm
I20240426 13:15:48.119052   398 webserver.cc:460] Webserver started at http://127.0.99.190:33807/ using document root <none> and password file <none>
I20240426 13:15:48.119179   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:48.119211   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:48.119257   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:48.119603   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "0f04336be0e6497085f02dd5fd932325"
format_stamp: "Formatted at 2024-04-26 13:15:48 on dist-test-slave-2jsl"
I20240426 13:15:48.120988   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:15:48.122033  8854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.122272   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:48.122350   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "0f04336be0e6497085f02dd5fd932325"
format_stamp: "Formatted at 2024-04-26 13:15:48 on dist-test-slave-2jsl"
I20240426 13:15:48.122417   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:48.132850   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:48.133080   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:48.136693   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:33783
I20240426 13:15:48.143015  8907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:48.143033  8906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:33783 every 8 connection(s)
I20240426 13:15:48.144680  8907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325: Bootstrap starting.
I20240426 13:15:48.145301  8907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:48.146219  8907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325: No bootstrap required, opened a new log
I20240426 13:15:48.146559  8907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f04336be0e6497085f02dd5fd932325" member_type: VOTER }
I20240426 13:15:48.146641  8907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:48.146674  8907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f04336be0e6497085f02dd5fd932325, State: Initialized, Role: FOLLOWER
I20240426 13:15:48.146793  8907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f04336be0e6497085f02dd5fd932325" member_type: VOTER }
I20240426 13:15:48.146858  8907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:15:48.146903  8907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:15:48.146966  8907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:48.147559  8907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f04336be0e6497085f02dd5fd932325" member_type: VOTER }
I20240426 13:15:48.147683  8907 leader_election.cc:302] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0f04336be0e6497085f02dd5fd932325; no voters: 
I20240426 13:15:48.147835  8907 leader_election.cc:288] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:15:48.147926  8910 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:15:48.148108  8910 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 1 LEADER]: Becoming Leader. State: Replica: 0f04336be0e6497085f02dd5fd932325, State: Running, Role: LEADER
I20240426 13:15:48.148188  8907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:15:48.148228  8910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f04336be0e6497085f02dd5fd932325" member_type: VOTER }
I20240426 13:15:48.148608  8912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0f04336be0e6497085f02dd5fd932325. Latest consensus state: current_term: 1 leader_uuid: "0f04336be0e6497085f02dd5fd932325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f04336be0e6497085f02dd5fd932325" member_type: VOTER } }
I20240426 13:15:48.148741  8912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [sys.catalog]: This master's current role is: LEADER
I20240426 13:15:48.148599  8911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0f04336be0e6497085f02dd5fd932325" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f04336be0e6497085f02dd5fd932325" member_type: VOTER } }
I20240426 13:15:48.148977  8911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [sys.catalog]: This master's current role is: LEADER
I20240426 13:15:48.149391  8916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:15:48.150003  8916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:15:48.150314   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:15:48.151572  8916 catalog_manager.cc:1347] Generated new cluster ID: dcb9a83600d34dbeb4fb7cce9e047b46
I20240426 13:15:48.151620  8916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:15:48.161011  8916 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:15:48.161444  8916 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:15:48.173568  8916 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325: Generated new TSK 0
I20240426 13:15:48.173697  8916 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:15:48.182927   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:48.184659  8928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:48.185523  8929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:48.186446  8931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:48.187083   398 server_base.cc:1034] running on GCE node
I20240426 13:15:48.187216   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:48.187268   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:48.187290   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137348187290 us; error 0 us; skew 500 ppm
I20240426 13:15:48.191629   398 webserver.cc:460] Webserver started at http://127.0.99.129:37649/ using document root <none> and password file <none>
I20240426 13:15:48.191749   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:48.191785   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:48.191848   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:48.192188   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "c6cdc666423a43ada231d9d3de151b68"
format_stamp: "Formatted at 2024-04-26 13:15:48 on dist-test-slave-2jsl"
I20240426 13:15:48.193483   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:48.194388  8936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.194612   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:15:48.194669   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "c6cdc666423a43ada231d9d3de151b68"
format_stamp: "Formatted at 2024-04-26 13:15:48 on dist-test-slave-2jsl"
I20240426 13:15:48.194733   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:48.207628   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:48.207859   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:48.208053   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:48.208436   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:48.208470   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.208510   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:48.208539   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.212090   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:36995
I20240426 13:15:48.212110  8999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:36995 every 8 connection(s)
I20240426 13:15:48.212709   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:48.214259  9004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:48.215133  9005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:48.215978  9007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:48.216491  9000 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33783
I20240426 13:15:48.216598  9000 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:48.216675   398 server_base.cc:1034] running on GCE node
I20240426 13:15:48.216846  9000 heartbeater.cc:511] Master 127.0.99.190:33783 requested a full tablet report, sending...
I20240426 13:15:48.216931   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:48.217000   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:48.217021   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137348217022 us; error 0 us; skew 500 ppm
I20240426 13:15:48.217423  8871 ts_manager.cc:194] Registered new tserver with Master: c6cdc666423a43ada231d9d3de151b68 (127.0.99.129:36995)
I20240426 13:15:48.217880   398 webserver.cc:460] Webserver started at http://127.0.99.130:33157/ using document root <none> and password file <none>
I20240426 13:15:48.218072   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:48.218113   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:48.218176  8871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57486
I20240426 13:15:48.218175   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:48.218732   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "177b2cace5c9472495b0895abefa11b9"
format_stamp: "Formatted at 2024-04-26 13:15:48 on dist-test-slave-2jsl"
I20240426 13:15:48.219969   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:15:48.220679  9012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.220861   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:15:48.220916   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "177b2cace5c9472495b0895abefa11b9"
format_stamp: "Formatted at 2024-04-26 13:15:48 on dist-test-slave-2jsl"
I20240426 13:15:48.220970   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:48.236510   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:48.236773   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:48.237020   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:48.237373   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:48.237408   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.237448   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:48.237486   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.241039   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:42035
I20240426 13:15:48.241070  9075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:42035 every 8 connection(s)
I20240426 13:15:48.241627   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:15:48.243065  9079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:48.243985  9080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:15:48.244714  9082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:15:48.245127  9076 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33783
I20240426 13:15:48.245195  9076 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:48.245383  9076 heartbeater.cc:511] Master 127.0.99.190:33783 requested a full tablet report, sending...
I20240426 13:15:48.245471   398 server_base.cc:1034] running on GCE node
I20240426 13:15:48.245644   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:15:48.245690   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:15:48.245711   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137348245711 us; error 0 us; skew 500 ppm
I20240426 13:15:48.246090  8871 ts_manager.cc:194] Registered new tserver with Master: 177b2cace5c9472495b0895abefa11b9 (127.0.99.130:42035)
I20240426 13:15:48.246609   398 webserver.cc:460] Webserver started at http://127.0.99.131:41701/ using document root <none> and password file <none>
I20240426 13:15:48.246721  8871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57494
I20240426 13:15:48.246745   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:15:48.246820   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:15:48.246887   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:15:48.247291   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "5271ad2d44bb4d029a96647cdfef4713"
format_stamp: "Formatted at 2024-04-26 13:15:48 on dist-test-slave-2jsl"
I20240426 13:15:48.248651   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:15:48.249465  9087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.249634   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:15:48.249696   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "5271ad2d44bb4d029a96647cdfef4713"
format_stamp: "Formatted at 2024-04-26 13:15:48 on dist-test-slave-2jsl"
I20240426 13:15:48.249758   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:15:48.257560   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:15:48.257786   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:15:48.258018   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:15:48.258388   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:15:48.258420   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.258457   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:15:48.258502   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:15:48.262192   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:41125
I20240426 13:15:48.262238  9150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:41125 every 8 connection(s)
I20240426 13:15:48.265503  9151 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33783
I20240426 13:15:48.265568  9151 heartbeater.cc:464] Registering TS with master...
I20240426 13:15:48.265692  9151 heartbeater.cc:511] Master 127.0.99.190:33783 requested a full tablet report, sending...
I20240426 13:15:48.266224  8871 ts_manager.cc:194] Registered new tserver with Master: 5271ad2d44bb4d029a96647cdfef4713 (127.0.99.131:41125)
I20240426 13:15:48.266647  8871 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57498
I20240426 13:15:48.266996   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004460495s
I20240426 13:15:48.267422   398 test_util.cc:268] Using random seed: -1279435363
I20240426 13:15:48.273897  8871 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57504:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:15:48.274273  8871 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:15:48.282063  9040 tablet_service.cc:1467] Processing CreateTablet for tablet 7cbefa1a175c40b1bce43345633e5c85 (DEFAULT_TABLE table=test-workload [id=9c54c28f182649b39b1a4b4e8f061523]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:48.282346  9040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cbefa1a175c40b1bce43345633e5c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:48.282485  8964 tablet_service.cc:1467] Processing CreateTablet for tablet 7cbefa1a175c40b1bce43345633e5c85 (DEFAULT_TABLE table=test-workload [id=9c54c28f182649b39b1a4b4e8f061523]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:48.282718  8964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cbefa1a175c40b1bce43345633e5c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:48.284579  9166 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: Bootstrap starting.
I20240426 13:15:48.285390  9166 tablet_bootstrap.cc:654] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:48.285514  9167 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Bootstrap starting.
I20240426 13:15:48.286242  9115 tablet_service.cc:1467] Processing CreateTablet for tablet 7cbefa1a175c40b1bce43345633e5c85 (DEFAULT_TABLE table=test-workload [id=9c54c28f182649b39b1a4b4e8f061523]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:15:48.286331  9166 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: No bootstrap required, opened a new log
I20240426 13:15:48.286418  9166 ts_tablet_manager.cc:1367] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:15:48.286463  9167 tablet_bootstrap.cc:654] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:48.286502  9115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cbefa1a175c40b1bce43345633e5c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:15:48.286767  9166 raft_consensus.cc:348] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.286864  9166 raft_consensus.cc:374] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:48.286899  9166 raft_consensus.cc:729] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177b2cace5c9472495b0895abefa11b9, State: Initialized, Role: FOLLOWER
I20240426 13:15:48.287014  9166 consensus_queue.cc:260] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.287385  9166 ts_tablet_manager.cc:1398] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:48.287393  9076 heartbeater.cc:503] Master 127.0.99.190:33783 was elected leader, sending a full tablet report...
I20240426 13:15:48.287650  9167 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: No bootstrap required, opened a new log
I20240426 13:15:48.287739  9167 ts_tablet_manager.cc:1367] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Time spent bootstrapping tablet: real 0.002s	user 0.002s	sys 0.000s
I20240426 13:15:48.288069  9167 raft_consensus.cc:348] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.288165  9167 raft_consensus.cc:374] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:48.288203  9167 raft_consensus.cc:729] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6cdc666423a43ada231d9d3de151b68, State: Initialized, Role: FOLLOWER
I20240426 13:15:48.288317  9167 consensus_queue.cc:260] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.288619  9167 ts_tablet_manager.cc:1398] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:48.288753  9000 heartbeater.cc:503] Master 127.0.99.190:33783 was elected leader, sending a full tablet report...
I20240426 13:15:48.288743  9171 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713: Bootstrap starting.
I20240426 13:15:48.289901  9171 tablet_bootstrap.cc:654] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713: Neither blocks nor log segments found. Creating new log.
I20240426 13:15:48.291177  9171 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713: No bootstrap required, opened a new log
I20240426 13:15:48.291242  9171 ts_tablet_manager.cc:1367] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:15:48.291523  9171 raft_consensus.cc:348] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.291599  9171 raft_consensus.cc:374] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:15:48.291626  9171 raft_consensus.cc:729] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5271ad2d44bb4d029a96647cdfef4713, State: Initialized, Role: FOLLOWER
I20240426 13:15:48.291703  9171 consensus_queue.cc:260] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.291937  9171 ts_tablet_manager.cc:1398] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:15:48.291990  9151 heartbeater.cc:503] Master 127.0.99.190:33783 was elected leader, sending a full tablet report...
I20240426 13:15:48.542788  9172 raft_consensus.cc:482] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:15:48.542917  9172 raft_consensus.cc:504] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.543324  9172 leader_election.cc:288] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 177b2cace5c9472495b0895abefa11b9 (127.0.99.130:42035), 5271ad2d44bb4d029a96647cdfef4713 (127.0.99.131:41125)
I20240426 13:15:48.546504  9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "c6cdc666423a43ada231d9d3de151b68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5271ad2d44bb4d029a96647cdfef4713" is_pre_election: true
I20240426 13:15:48.546504  9050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "c6cdc666423a43ada231d9d3de151b68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177b2cace5c9472495b0895abefa11b9" is_pre_election: true
I20240426 13:15:48.546676  9050 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6cdc666423a43ada231d9d3de151b68 in term 0.
I20240426 13:15:48.546676  9125 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6cdc666423a43ada231d9d3de151b68 in term 0.
I20240426 13:15:48.547015  8937 leader_election.cc:302] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177b2cace5c9472495b0895abefa11b9, c6cdc666423a43ada231d9d3de151b68; no voters: 
I20240426 13:15:48.547154  9172 raft_consensus.cc:2781] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:15:48.547206  9172 raft_consensus.cc:482] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:15:48.547252  9172 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:48.548058  9172 raft_consensus.cc:504] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.548331  9172 leader_election.cc:288] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [CANDIDATE]: Term 1 election: Requested vote from peers 177b2cace5c9472495b0895abefa11b9 (127.0.99.130:42035), 5271ad2d44bb4d029a96647cdfef4713 (127.0.99.131:41125)
I20240426 13:15:48.548580  9050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "c6cdc666423a43ada231d9d3de151b68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177b2cace5c9472495b0895abefa11b9"
I20240426 13:15:48.548589  9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "c6cdc666423a43ada231d9d3de151b68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5271ad2d44bb4d029a96647cdfef4713"
I20240426 13:15:48.548698  9125 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:48.548698  9050 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:15:48.549702  9125 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6cdc666423a43ada231d9d3de151b68 in term 1.
I20240426 13:15:48.549726  9050 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6cdc666423a43ada231d9d3de151b68 in term 1.
I20240426 13:15:48.549989  8937 leader_election.cc:302] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177b2cace5c9472495b0895abefa11b9, c6cdc666423a43ada231d9d3de151b68; no voters: 
I20240426 13:15:48.550171  9172 raft_consensus.cc:2781] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:15:48.550341  9172 raft_consensus.cc:686] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 LEADER]: Becoming Leader. State: Replica: c6cdc666423a43ada231d9d3de151b68, State: Running, Role: LEADER
I20240426 13:15:48.550484  9172 consensus_queue.cc:237] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:15:48.551877  8871 catalog_manager.cc:5484] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6cdc666423a43ada231d9d3de151b68 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "c6cdc666423a43ada231d9d3de151b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } health_report { overall_health: UNKNOWN } } }
I20240426 13:15:48.586311  9050 raft_consensus.cc:1261] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 1 FOLLOWER]: Refusing update from remote peer c6cdc666423a43ada231d9d3de151b68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:15:48.586320  9125 raft_consensus.cc:1261] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 1 FOLLOWER]: Refusing update from remote peer c6cdc666423a43ada231d9d3de151b68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:15:48.586736  9176 consensus_queue.cc:1035] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:15:48.586936  9172 consensus_queue.cc:1035] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:15:58.584545   398 tablet_replica.cc:330] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: stopping tablet replica
I20240426 13:15:58.584647   398 raft_consensus.cc:2229] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:15:58.584801   398 raft_consensus.cc:2258] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:15:58.588618   398 ts_tablet_manager.cc:1875] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20240426 13:15:58.590384   398 ts_tablet_manager.cc:1888] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3151
I20240426 13:15:58.590456   398 log.cc:1196] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Deleting WAL directory at /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound.1714137102075184-398-0/minicluster-data/ts-0-root/wals/7cbefa1a175c40b1bce43345633e5c85
I20240426 13:16:00.071211  9237 raft_consensus.cc:482] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6cdc666423a43ada231d9d3de151b68)
I20240426 13:16:00.071342  9237 raft_consensus.cc:504] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:00.071854  9237 leader_election.cc:288] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6cdc666423a43ada231d9d3de151b68 (127.0.99.129:36995), 5271ad2d44bb4d029a96647cdfef4713 (127.0.99.131:41125)
I20240426 13:16:00.072760  9242 raft_consensus.cc:482] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c6cdc666423a43ada231d9d3de151b68)
I20240426 13:16:00.072913  9242 raft_consensus.cc:504] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:00.073314  9242 leader_election.cc:288] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 177b2cace5c9472495b0895abefa11b9 (127.0.99.130:42035), c6cdc666423a43ada231d9d3de151b68 (127.0.99.129:36995)
I20240426 13:16:00.075372  8974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "177b2cace5c9472495b0895abefa11b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3151 } } ignore_live_leader: false dest_uuid: "c6cdc666423a43ada231d9d3de151b68" is_pre_election: true
I20240426 13:16:00.075507  8974 raft_consensus.cc:1789] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: voting while tombstoned based on last-logged opid 1.3151
I20240426 13:16:00.075613  8974 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177b2cace5c9472495b0895abefa11b9 in term 1.
I20240426 13:16:00.075819  9124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "177b2cace5c9472495b0895abefa11b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3151 } } ignore_live_leader: false dest_uuid: "5271ad2d44bb4d029a96647cdfef4713" is_pre_election: true
I20240426 13:16:00.075848  9013 leader_election.cc:302] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177b2cace5c9472495b0895abefa11b9, c6cdc666423a43ada231d9d3de151b68; no voters: 
I20240426 13:16:00.075919  9124 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 177b2cace5c9472495b0895abefa11b9 in term 1.
I20240426 13:16:00.076004  9237 raft_consensus.cc:2781] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20240426 13:16:00.076058  9237 raft_consensus.cc:482] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c6cdc666423a43ada231d9d3de151b68)
I20240426 13:16:00.076110  9237 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:00.077175  9237 raft_consensus.cc:504] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:00.077358  8974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "5271ad2d44bb4d029a96647cdfef4713" candidate_term: 2 candidate_status { last_received { term: 1 index: 3151 } } ignore_live_leader: false dest_uuid: "c6cdc666423a43ada231d9d3de151b68" is_pre_election: true
I20240426 13:16:00.077438  8974 raft_consensus.cc:1789] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: voting while tombstoned based on last-logged opid 1.3151
I20240426 13:16:00.077500  8974 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5271ad2d44bb4d029a96647cdfef4713 in term 1.
I20240426 13:16:00.077564  9237 leader_election.cc:288] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [CANDIDATE]: Term 2 election: Requested vote from peers c6cdc666423a43ada231d9d3de151b68 (127.0.99.129:36995), 5271ad2d44bb4d029a96647cdfef4713 (127.0.99.131:41125)
I20240426 13:16:00.077935  9124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "177b2cace5c9472495b0895abefa11b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3151 } } ignore_live_leader: false dest_uuid: "5271ad2d44bb4d029a96647cdfef4713"
I20240426 13:16:00.077940  8974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "177b2cace5c9472495b0895abefa11b9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3151 } } ignore_live_leader: false dest_uuid: "c6cdc666423a43ada231d9d3de151b68"
I20240426 13:16:00.078080  9124 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:00.078104  8974 raft_consensus.cc:1789] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: voting while tombstoned based on last-logged opid 1.3151
I20240426 13:16:00.078178  8974 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:00.078267  9088 leader_election.cc:302] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5271ad2d44bb4d029a96647cdfef4713, c6cdc666423a43ada231d9d3de151b68; no voters: 
I20240426 13:16:00.078912  9124 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177b2cace5c9472495b0895abefa11b9 in term 2.
I20240426 13:16:00.078991  8974 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 177b2cace5c9472495b0895abefa11b9 in term 2.
I20240426 13:16:00.079013  9242 raft_consensus.cc:2741] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20240426 13:16:00.079056  9050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "5271ad2d44bb4d029a96647cdfef4713" candidate_term: 2 candidate_status { last_received { term: 1 index: 3151 } } ignore_live_leader: false dest_uuid: "177b2cace5c9472495b0895abefa11b9" is_pre_election: true
I20240426 13:16:00.079172  9050 raft_consensus.cc:2379] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5271ad2d44bb4d029a96647cdfef4713 in current term 2: Already voted for candidate 177b2cace5c9472495b0895abefa11b9 in this term.
I20240426 13:16:00.079201  9015 leader_election.cc:302] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177b2cace5c9472495b0895abefa11b9, 5271ad2d44bb4d029a96647cdfef4713; no voters: 
I20240426 13:16:00.079398  9237 raft_consensus.cc:2781] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:16:00.079612  9237 raft_consensus.cc:686] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 LEADER]: Becoming Leader. State: Replica: 177b2cace5c9472495b0895abefa11b9, State: Running, Role: LEADER
I20240426 13:16:00.079789  9237 consensus_queue.cc:237] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3151, Committed index: 3151, Last appended: 1.3151, Last appended by leader: 3151, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:00.081508  8864 catalog_manager.cc:5484] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 reported cstate change: term changed from 1 to 2, leader changed from c6cdc666423a43ada231d9d3de151b68 (127.0.99.129) to 177b2cace5c9472495b0895abefa11b9 (127.0.99.130). New cstate: current_term: 2 leader_uuid: "177b2cace5c9472495b0895abefa11b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:00.090497  9124 raft_consensus.cc:1261] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 2 FOLLOWER]: Refusing update from remote peer 177b2cace5c9472495b0895abefa11b9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3151. Preceding OpId from leader: term: 2 index: 3153. (index mismatch)
W20240426 13:16:00.090649  9013 consensus_peers.cc:485] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 -> Peer c6cdc666423a43ada231d9d3de151b68 (127.0.99.129:36995): Couldn't send request to peer c6cdc666423a43ada231d9d3de151b68. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20240426 13:16:00.090847  9237 consensus_queue.cc:1035] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3152, Last known committed idx: 3151, Time since last communication: 0.000s
I20240426 13:16:00.484527  9261 ts_tablet_manager.cc:902] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Initiating tablet copy from peer 177b2cace5c9472495b0895abefa11b9 (127.0.99.130:42035)
I20240426 13:16:00.485338  9261 tablet_copy_client.cc:308] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: tablet copy: Beginning tablet copy session from remote peer at address 127.0.99.130:42035
I20240426 13:16:00.485833  9060 tablet_copy_service.cc:140] P 177b2cace5c9472495b0895abefa11b9: Received BeginTabletCopySession request for tablet 7cbefa1a175c40b1bce43345633e5c85 from peer c6cdc666423a43ada231d9d3de151b68 ({username='slave'} at 127.0.0.1:60756)
I20240426 13:16:00.485936  9060 tablet_copy_service.cc:161] P 177b2cace5c9472495b0895abefa11b9: Beginning new tablet copy session on tablet 7cbefa1a175c40b1bce43345633e5c85 from peer c6cdc666423a43ada231d9d3de151b68 at {username='slave'} at 127.0.0.1:60756: session id = c6cdc666423a43ada231d9d3de151b68-7cbefa1a175c40b1bce43345633e5c85
I20240426 13:16:00.487257  9060 tablet_copy_source_session.cc:205] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: Tablet Copy: opened 0 blocks and 1 log segments
I20240426 13:16:00.488466  9261 ts_tablet_manager.cc:1875] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Deleting tablet data with delete state TABLET_DATA_COPYING
I20240426 13:16:00.490862  9261 ts_tablet_manager.cc:1888] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.3151
I20240426 13:16:00.491042  9261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cbefa1a175c40b1bce43345633e5c85. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:00.492120  9261 tablet_copy_client.cc:783] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: tablet copy: Starting download of 0 data blocks...
I20240426 13:16:00.492300  9261 tablet_copy_client.cc:647] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: tablet copy: Starting download of 1 WAL segments...
I20240426 13:16:00.514657  9261 tablet_copy_client.cc:523] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20240426 13:16:00.515908  9261 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Bootstrap starting.
W20240426 13:16:03.082875  9013 consensus_peers.cc:485] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 -> Peer c6cdc666423a43ada231d9d3de151b68 (127.0.99.129:36995): Couldn't send request to peer c6cdc666423a43ada231d9d3de151b68. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20240426 13:16:05.237982  9261 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Bootstrap replayed 1/1 log segments. Stats: ops{read=3339 overwritten=0 applied=3339 ignored=0} inserts{seen=157500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:16:05.238229  9261 tablet_bootstrap.cc:492] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Bootstrap complete.
I20240426 13:16:05.239284  9261 ts_tablet_manager.cc:1367] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Time spent bootstrapping tablet: real 4.723s	user 4.633s	sys 0.088s
I20240426 13:16:05.239617  9261 raft_consensus.cc:348] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:05.239714  9261 raft_consensus.cc:729] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6cdc666423a43ada231d9d3de151b68, State: Initialized, Role: FOLLOWER
I20240426 13:16:05.239818  9261 consensus_queue.cc:260] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3339, Last appended: 2.3339, Last appended by leader: 3339, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:05.240067  9261 ts_tablet_manager.cc:1398] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:16:05.240574  9060 tablet_copy_service.cc:342] P 177b2cace5c9472495b0895abefa11b9: Request end of tablet copy session c6cdc666423a43ada231d9d3de151b68-7cbefa1a175c40b1bce43345633e5c85 received from {username='slave'} at 127.0.0.1:60756
I20240426 13:16:05.240664  9060 tablet_copy_service.cc:433] P 177b2cace5c9472495b0895abefa11b9: ending tablet copy session c6cdc666423a43ada231d9d3de151b68-7cbefa1a175c40b1bce43345633e5c85 on tablet 7cbefa1a175c40b1bce43345633e5c85 with peer c6cdc666423a43ada231d9d3de151b68
I20240426 13:16:05.441241  8974 raft_consensus.cc:1203] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.3151->[2.3152-2.3826]   Dedup: 2.3339->[2.3340-2.3826]
I20240426 13:16:05.490950  9286 raft_consensus.cc:979] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: : Instructing follower 5271ad2d44bb4d029a96647cdfef4713 to start an election
I20240426 13:16:05.491052  9286 raft_consensus.cc:1067] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 LEADER]: Signalling peer 5271ad2d44bb4d029a96647cdfef4713 to start an election
I20240426 13:16:05.491513  9125 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85"
dest_uuid: "5271ad2d44bb4d029a96647cdfef4713"
 from {username='slave'} at 127.0.0.1:60240
I20240426 13:16:05.491811  9286 raft_consensus.cc:979] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: : Instructing follower c6cdc666423a43ada231d9d3de151b68 to start an election
I20240426 13:16:05.491897  9286 raft_consensus.cc:1067] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 LEADER]: Signalling peer c6cdc666423a43ada231d9d3de151b68 to start an election
W20240426 13:16:05.491873  9015 consensus_peers.cc:330] unable to start election on peer 5271ad2d44bb4d029a96647cdfef4713: Illegal state: leader elections are disabled
I20240426 13:16:05.492260  8974 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85"
dest_uuid: "c6cdc666423a43ada231d9d3de151b68"
 from {username='slave'} at 127.0.0.1:39798
I20240426 13:16:05.492357  8974 raft_consensus.cc:482] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20240426 13:16:05.492413  8974 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:16:05.493356  8974 raft_consensus.cc:504] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:05.493805  8974 leader_election.cc:288] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [CANDIDATE]: Term 3 election: Requested vote from peers 177b2cace5c9472495b0895abefa11b9 (127.0.99.130:42035), 5271ad2d44bb4d029a96647cdfef4713 (127.0.99.131:41125)
I20240426 13:16:05.494016  9050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "c6cdc666423a43ada231d9d3de151b68" candidate_term: 3 candidate_status { last_received { term: 2 index: 3845 } } ignore_live_leader: true dest_uuid: "177b2cace5c9472495b0895abefa11b9"
I20240426 13:16:05.494134  9050 raft_consensus.cc:3032] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 LEADER]: Stepping down as leader of term 2
I20240426 13:16:05.494171  9050 raft_consensus.cc:729] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 177b2cace5c9472495b0895abefa11b9, State: Running, Role: LEADER
I20240426 13:16:05.494233  9125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cbefa1a175c40b1bce43345633e5c85" candidate_uuid: "c6cdc666423a43ada231d9d3de151b68" candidate_term: 3 candidate_status { last_received { term: 2 index: 3845 } } ignore_live_leader: true dest_uuid: "5271ad2d44bb4d029a96647cdfef4713"
I20240426 13:16:05.494341  9125 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:16:05.494320  9050 consensus_queue.cc:260] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3845, Committed index: 3845, Last appended: 2.3845, Last appended by leader: 3845, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:05.494465  9050 raft_consensus.cc:3037] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:16:05.495128  9125 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6cdc666423a43ada231d9d3de151b68 in term 3.
I20240426 13:16:05.495313  9050 raft_consensus.cc:2454] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6cdc666423a43ada231d9d3de151b68 in term 3.
I20240426 13:16:05.495373  8939 leader_election.cc:302] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5271ad2d44bb4d029a96647cdfef4713, c6cdc666423a43ada231d9d3de151b68; no voters: 
I20240426 13:16:05.495553  9280 raft_consensus.cc:2781] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 3 FOLLOWER]: Leader election won for term 3
I20240426 13:16:05.495678  9280 raft_consensus.cc:686] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 3 LEADER]: Becoming Leader. State: Replica: c6cdc666423a43ada231d9d3de151b68, State: Running, Role: LEADER
I20240426 13:16:05.495810  9280 consensus_queue.cc:237] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3845, Committed index: 3845, Last appended: 2.3845, Last appended by leader: 3845, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } }
I20240426 13:16:05.496981  8863 catalog_manager.cc:5484] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 reported cstate change: term changed from 2 to 3, leader changed from 177b2cace5c9472495b0895abefa11b9 (127.0.99.130) to c6cdc666423a43ada231d9d3de151b68 (127.0.99.129). New cstate: current_term: 3 leader_uuid: "c6cdc666423a43ada231d9d3de151b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "177b2cace5c9472495b0895abefa11b9" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 42035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6cdc666423a43ada231d9d3de151b68" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 36995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5271ad2d44bb4d029a96647cdfef4713" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 41125 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:05.625643   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:16:05.628691   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:05.628877   398 tablet_replica.cc:330] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: stopping tablet replica
I20240426 13:16:05.628993   398 raft_consensus.cc:2229] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 3 LEADER]: Raft consensus shutting down.
I20240426 13:16:05.629388   398 pending_rounds.cc:62] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Trying to abort 1 pending ops.
I20240426 13:16:05.629422   398 pending_rounds.cc:68] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68: Aborting op as it isn't in flight: id { term: 3 index: 3846 } timestamp: 7021106649072246784 op_type: NO_OP noop_request { }
I20240426 13:16:05.629498   398 raft_consensus.cc:2866] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20240426 13:16:05.629565   398 raft_consensus.cc:2258] T 7cbefa1a175c40b1bce43345633e5c85 P c6cdc666423a43ada231d9d3de151b68 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:05.647974   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:16:05.651247   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:16:05.655078   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:05.655239   398 tablet_replica.cc:330] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9: stopping tablet replica
I20240426 13:16:05.655308   398 raft_consensus.cc:2229] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:05.655375   398 raft_consensus.cc:2258] T 7cbefa1a175c40b1bce43345633e5c85 P 177b2cace5c9472495b0895abefa11b9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:05.673719   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:16:05.675385   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:16:05.678211   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:05.678352   398 tablet_replica.cc:330] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713: stopping tablet replica
I20240426 13:16:05.678436   398 raft_consensus.cc:2229] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 3 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:05.678534   398 raft_consensus.cc:2258] T 7cbefa1a175c40b1bce43345633e5c85 P 5271ad2d44bb4d029a96647cdfef4713 [term 3 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:05.696050   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:16:05.697607   398 master.cc:537] Master@127.0.99.190:33783 shutting down...
I20240426 13:16:05.700489   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:16:05.700580   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:05.700619   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0f04336be0e6497085f02dd5fd932325: stopping tablet replica
I20240426 13:16:05.712631   398 master.cc:552] Master@127.0.99.190:33783 shutdown complete.
[       OK ] TxnParticipantITest.TestTxnSystemClientRetriesWhenReplicaNotFound (17607 ms)
[----------] 21 tests from TxnParticipantITest (263636 ms total)

[----------] 2 tests from TxnParticipantElectionStormITest
[ RUN      ] TxnParticipantElectionStormITest.TestFrequentElections
I20240426 13:16:05.720244   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:35049
I20240426 13:16:05.720503   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:05.722715  9297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:05.723618  9298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:05.724396  9300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:05.725263   398 server_base.cc:1034] running on GCE node
I20240426 13:16:05.725390   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:05.725425   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:05.725440   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137365725439 us; error 0 us; skew 500 ppm
I20240426 13:16:05.726260   398 webserver.cc:460] Webserver started at http://127.0.99.190:43941/ using document root <none> and password file <none>
I20240426 13:16:05.726366   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:05.726398   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:05.726436   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:05.726745   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "5bd9e4fe7a574d059ee98e460d03b15e"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:05.728029   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:16:05.728931  9305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.729142   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:05.729199   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "5bd9e4fe7a574d059ee98e460d03b15e"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:05.729260   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:05.750177   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:05.750401   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:05.753662   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:35049
I20240426 13:16:05.760695  9357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:35049 every 8 connection(s)
I20240426 13:16:05.761047  9358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:05.762473  9358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: Bootstrap starting.
I20240426 13:16:05.763042  9358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:05.763774  9358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: No bootstrap required, opened a new log
I20240426 13:16:05.764053  9358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER }
I20240426 13:16:05.764124  9358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:05.764155  9358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd9e4fe7a574d059ee98e460d03b15e, State: Initialized, Role: FOLLOWER
I20240426 13:16:05.764276  9358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER }
I20240426 13:16:05.764340  9358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:16:05.764376  9358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:16:05.764420  9358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:05.764967  9358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER }
I20240426 13:16:05.765089  9358 leader_election.cc:302] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bd9e4fe7a574d059ee98e460d03b15e; no voters: 
I20240426 13:16:05.765230  9358 leader_election.cc:288] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:16:05.765311  9361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:16:05.765489  9361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 LEADER]: Becoming Leader. State: Replica: 5bd9e4fe7a574d059ee98e460d03b15e, State: Running, Role: LEADER
I20240426 13:16:05.765575  9358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:16:05.765619  9361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER }
I20240426 13:16:05.765990  9362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER } }
I20240426 13:16:05.766014  9363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bd9e4fe7a574d059ee98e460d03b15e. Latest consensus state: current_term: 1 leader_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER } }
I20240426 13:16:05.766202  9363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: This master's current role is: LEADER
I20240426 13:16:05.766418  9362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: This master's current role is: LEADER
I20240426 13:16:05.766595  9368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:16:05.767112  9368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:16:05.767278   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:16:05.768977  9368 catalog_manager.cc:1347] Generated new cluster ID: 7b0675963f064e5bbcf304642f0ca1c8
I20240426 13:16:05.769023  9368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:16:05.801605  9368 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:16:05.802071  9368 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:16:05.808200  9368 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: Generated new TSK 0
I20240426 13:16:05.808336  9368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:16:05.831426   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:05.833586  9380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:05.834331  9382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:05.835065  9379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:05.835783   398 server_base.cc:1034] running on GCE node
I20240426 13:16:05.835906   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:05.835939   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:05.835958   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137365835958 us; error 0 us; skew 500 ppm
I20240426 13:16:05.836669   398 webserver.cc:460] Webserver started at http://127.0.99.129:33223/ using document root <none> and password file <none>
I20240426 13:16:05.836795   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:05.836841   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:05.836910   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:05.837196   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "5606172f13e44bd1a9372805a8e67d8d"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:05.838546   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:16:05.839360  9387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.839545   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:05.839617   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "5606172f13e44bd1a9372805a8e67d8d"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:05.839675   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:05.856662   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:05.856886   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:05.857086   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:05.857425   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:05.857458   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.857514   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:05.857543   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.861032   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:35405
I20240426 13:16:05.861052  9450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:35405 every 8 connection(s)
I20240426 13:16:05.861622   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:05.863658  9455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:05.864356  9458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:05.865041   398 server_base.cc:1034] running on GCE node
I20240426 13:16:05.865257  9451 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35049
I20240426 13:16:05.865326  9451 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:05.865484  9451 heartbeater.cc:511] Master 127.0.99.190:35049 requested a full tablet report, sending...
W20240426 13:16:05.865682  9456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:05.865895   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:05.865952   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:05.865985   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137365865985 us; error 0 us; skew 500 ppm
I20240426 13:16:05.866099  9322 ts_manager.cc:194] Registered new tserver with Master: 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405)
I20240426 13:16:05.866845   398 webserver.cc:460] Webserver started at http://127.0.99.130:43269/ using document root <none> and password file <none>
I20240426 13:16:05.866943  9322 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52240
I20240426 13:16:05.866971   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:05.867008   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:05.867063   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:05.867391   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "469ce5ad20d6420da45e9159560fbe64"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:05.868711   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:05.869411  9463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.869601   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:16:05.869655   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "469ce5ad20d6420da45e9159560fbe64"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:05.869714   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:05.877069   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:05.877285   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:05.877555   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:05.877907   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:05.877943   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.878005   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:05.878043   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.881263   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:35953
I20240426 13:16:05.881287  9526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:35953 every 8 connection(s)
I20240426 13:16:05.881814   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:05.883802  9530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:05.884459  9531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:05.885188  9533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:05.885246  9527 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35049
I20240426 13:16:05.885314  9527 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:05.885447  9527 heartbeater.cc:511] Master 127.0.99.190:35049 requested a full tablet report, sending...
I20240426 13:16:05.885934   398 server_base.cc:1034] running on GCE node
I20240426 13:16:05.886073  9322 ts_manager.cc:194] Registered new tserver with Master: 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953)
I20240426 13:16:05.886090   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:05.886138   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:05.886159   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137365886159 us; error 0 us; skew 500 ppm
I20240426 13:16:05.886512  9322 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52256
I20240426 13:16:05.886976   398 webserver.cc:460] Webserver started at http://127.0.99.131:35881/ using document root <none> and password file <none>
I20240426 13:16:05.887099   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:05.887151   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:05.887207   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:05.887455   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "a08072d49eb746bda23d4045f3b6de1f"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:05.888659   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:16:05.889288  9538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.889456   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:16:05.889519   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "a08072d49eb746bda23d4045f3b6de1f"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:05.889572   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:05.903019   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:05.903244   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:05.903447   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:05.903784   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:05.903828   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.903867   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:05.903894   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:05.907488   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:35375
I20240426 13:16:05.907548  9601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:35375 every 8 connection(s)
I20240426 13:16:05.911301  9602 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35049
I20240426 13:16:05.911362  9602 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:05.911516  9602 heartbeater.cc:511] Master 127.0.99.190:35049 requested a full tablet report, sending...
I20240426 13:16:05.911973  9322 ts_manager.cc:194] Registered new tserver with Master: a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
I20240426 13:16:05.912302   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004447516s
I20240426 13:16:05.912425  9322 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52268
I20240426 13:16:05.912778   398 test_util.cc:268] Using random seed: -1261790007
I20240426 13:16:05.918788  9322 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52284:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:16:05.919102  9322 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:16:05.927134  9491 tablet_service.cc:1467] Processing CreateTablet for tablet 90b65ca9cfe8400baaff49156ed2a3d2 (DEFAULT_TABLE table=test-workload [id=de2465e79a614b82bfc0c2df1d1471c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:05.927368  9491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b65ca9cfe8400baaff49156ed2a3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:05.927317  9415 tablet_service.cc:1467] Processing CreateTablet for tablet 90b65ca9cfe8400baaff49156ed2a3d2 (DEFAULT_TABLE table=test-workload [id=de2465e79a614b82bfc0c2df1d1471c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:05.927593  9415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b65ca9cfe8400baaff49156ed2a3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:05.928040  9566 tablet_service.cc:1467] Processing CreateTablet for tablet 90b65ca9cfe8400baaff49156ed2a3d2 (DEFAULT_TABLE table=test-workload [id=de2465e79a614b82bfc0c2df1d1471c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:05.928261  9566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90b65ca9cfe8400baaff49156ed2a3d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:05.929440  9617 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Bootstrap starting.
I20240426 13:16:05.929944  9618 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Bootstrap starting.
I20240426 13:16:05.930297  9619 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Bootstrap starting.
I20240426 13:16:05.930366  9617 tablet_bootstrap.cc:654] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:05.930851  9618 tablet_bootstrap.cc:654] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:05.930958  9619 tablet_bootstrap.cc:654] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:05.931742  9617 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: No bootstrap required, opened a new log
I20240426 13:16:05.931809  9618 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: No bootstrap required, opened a new log
I20240426 13:16:05.931905  9618 ts_tablet_manager.cc:1367] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Time spent bootstrapping tablet: real 0.002s	user 0.000s	sys 0.002s
I20240426 13:16:05.932066  9619 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: No bootstrap required, opened a new log
I20240426 13:16:05.932132  9619 ts_tablet_manager.cc:1367] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:16:05.932260  9618 raft_consensus.cc:348] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.932338  9618 raft_consensus.cc:374] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:05.932361  9618 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5606172f13e44bd1a9372805a8e67d8d, State: Initialized, Role: FOLLOWER
I20240426 13:16:05.932451  9619 raft_consensus.cc:348] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.932538  9619 raft_consensus.cc:374] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:05.932478  9618 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.932574  9619 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08072d49eb746bda23d4045f3b6de1f, State: Initialized, Role: FOLLOWER
I20240426 13:16:05.932686  9619 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.932821  9451 heartbeater.cc:503] Master 127.0.99.190:35049 was elected leader, sending a full tablet report...
I20240426 13:16:05.932844  9618 ts_tablet_manager.cc:1398] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:05.931819  9617 ts_tablet_manager.cc:1367] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Time spent bootstrapping tablet: real 0.002s	user 0.000s	sys 0.002s
I20240426 13:16:05.932967  9619 ts_tablet_manager.cc:1398] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:05.933101  9602 heartbeater.cc:503] Master 127.0.99.190:35049 was elected leader, sending a full tablet report...
I20240426 13:16:05.933275  9617 raft_consensus.cc:348] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.933377  9617 raft_consensus.cc:374] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:05.933418  9617 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469ce5ad20d6420da45e9159560fbe64, State: Initialized, Role: FOLLOWER
I20240426 13:16:05.933543  9617 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.933841  9527 heartbeater.cc:503] Master 127.0.99.190:35049 was elected leader, sending a full tablet report...
I20240426 13:16:05.933866  9617 ts_tablet_manager.cc:1398] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:05.934172  9623 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:16:05.934255  9623 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.934670  9623 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
I20240426 13:16:05.938413  9501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "5606172f13e44bd1a9372805a8e67d8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469ce5ad20d6420da45e9159560fbe64" is_pre_election: true
I20240426 13:16:05.938520  9501 raft_consensus.cc:2454] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5606172f13e44bd1a9372805a8e67d8d in term 0.
I20240426 13:16:05.939008  9389 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d; no voters: 
I20240426 13:16:05.939172  9623 raft_consensus.cc:2781] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:16:05.939172  9576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "5606172f13e44bd1a9372805a8e67d8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a08072d49eb746bda23d4045f3b6de1f" is_pre_election: true
I20240426 13:16:05.939246  9623 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:05.939311  9623 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:05.939332  9576 raft_consensus.cc:2454] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5606172f13e44bd1a9372805a8e67d8d in term 0.
I20240426 13:16:05.940025  9623 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.940300  9623 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 1 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
I20240426 13:16:05.940521  9501 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "5606172f13e44bd1a9372805a8e67d8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "469ce5ad20d6420da45e9159560fbe64"
I20240426 13:16:05.940564  9576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "5606172f13e44bd1a9372805a8e67d8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a08072d49eb746bda23d4045f3b6de1f"
I20240426 13:16:05.940605  9501 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:05.940646  9576 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:05.941241  9501 raft_consensus.cc:2454] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5606172f13e44bd1a9372805a8e67d8d in term 1.
I20240426 13:16:05.941462  9389 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d; no voters: 
I20240426 13:16:05.941599  9623 raft_consensus.cc:2781] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:16:05.941632  9576 raft_consensus.cc:2454] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5606172f13e44bd1a9372805a8e67d8d in term 1.
I20240426 13:16:05.941812  9623 raft_consensus.cc:686] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 1 LEADER]: Becoming Leader. State: Replica: 5606172f13e44bd1a9372805a8e67d8d, State: Running, Role: LEADER
I20240426 13:16:05.941957  9623 consensus_queue.cc:237] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:05.943230  9322 catalog_manager.cc:5484] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129). New cstate: current_term: 1 leader_uuid: "5606172f13e44bd1a9372805a8e67d8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:05.965268  9576 raft_consensus.cc:1261] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 1 FOLLOWER]: Refusing update from remote peer 5606172f13e44bd1a9372805a8e67d8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:16:05.965384  9501 raft_consensus.cc:1261] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 1 FOLLOWER]: Refusing update from remote peer 5606172f13e44bd1a9372805a8e67d8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:16:05.965633  9627 consensus_queue.cc:1035] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:16:05.965775  9623 consensus_queue.cc:1035] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:16:15.982401  9680 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5606172f13e44bd1a9372805a8e67d8d)
I20240426 13:16:15.982487  9680 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:15.983246  9680 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:15.983433  9575 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 2 FOLLOWER]: Rejecting Update request from peer 5606172f13e44bd1a9372805a8e67d8d for earlier term 1. Current term is 2. Ops: []
I20240426 13:16:15.983686  9680 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 2 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405)
I20240426 13:16:15.983929  9664 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: INVALID_TERM, Last received: 1.3043, Next index: 3044, Last known committed idx: 3043, Time since last communication: 0.000s
W20240426 13:16:15.984124  9664 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20240426 13:16:15.984161  9664 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20240426 13:16:15.987217  9500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3043 } } ignore_live_leader: false dest_uuid: "469ce5ad20d6420da45e9159560fbe64"
I20240426 13:16:15.987284  9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3043 } } ignore_live_leader: false dest_uuid: "5606172f13e44bd1a9372805a8e67d8d"
I20240426 13:16:15.987586  9541 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a08072d49eb746bda23d4045f3b6de1f; no voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d
I20240426 13:16:15.987722  9680 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240426 13:16:16.005045  9575 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 2 FOLLOWER]: Rejecting Update request from peer 5606172f13e44bd1a9372805a8e67d8d for earlier term 1. Current term is 2. Ops: []
I20240426 13:16:16.005340  9623 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: INVALID_TERM, Last received: 1.3043, Next index: 3044, Last known committed idx: 3043, Time since last communication: 0.000s
I20240426 13:16:16.007331  9664 raft_consensus.cc:3032] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:16:16.007391  9664 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5606172f13e44bd1a9372805a8e67d8d, State: Running, Role: LEADER
I20240426 13:16:16.007503  9664 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3043, Committed index: 3043, Last appended: 1.3043, Last appended by leader: 3043, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.007673  9664 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:16.029850  9623 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:16.029922  9623 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:16:16.030330  9680 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:16.030397  9680 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:16:16.030405  9850 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5606172f13e44bd1a9372805a8e67d8d)
I20240426 13:16:16.030489  9850 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:16.030803  9623 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.031070  9623 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 3 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
I20240426 13:16:16.031224  9850 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.031306  9500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "5606172f13e44bd1a9372805a8e67d8d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3043 } } ignore_live_leader: false dest_uuid: "469ce5ad20d6420da45e9159560fbe64"
I20240426 13:16:16.031431  9500 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:16:16.031515  9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "5606172f13e44bd1a9372805a8e67d8d" candidate_term: 3 candidate_status { last_received { term: 1 index: 3043 } } ignore_live_leader: false dest_uuid: "a08072d49eb746bda23d4045f3b6de1f"
I20240426 13:16:16.032313  9500 raft_consensus.cc:2454] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5606172f13e44bd1a9372805a8e67d8d in term 3.
I20240426 13:16:16.032402  9850 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 2 election: Requested vote from peers 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
I20240426 13:16:16.032927  9389 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d; no voters: 
I20240426 13:16:16.033071  9623 raft_consensus.cc:2781] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 3 FOLLOWER]: Leader election won for term 3
I20240426 13:16:16.033144  9623 raft_consensus.cc:686] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 3 LEADER]: Becoming Leader. State: Replica: 5606172f13e44bd1a9372805a8e67d8d, State: Running, Role: LEADER
I20240426 13:16:16.033268  9623 consensus_queue.cc:237] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3043, Committed index: 3043, Last appended: 1.3043, Last appended by leader: 3043, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.031224  9680 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.034204  9680 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 3 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405)
I20240426 13:16:16.034320  9575 raft_consensus.cc:2379] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5606172f13e44bd1a9372805a8e67d8d in current term 3: Already voted for candidate a08072d49eb746bda23d4045f3b6de1f in this term.
I20240426 13:16:16.034919  9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 3 candidate_status { last_received { term: 1 index: 3043 } } ignore_live_leader: false dest_uuid: "5606172f13e44bd1a9372805a8e67d8d"
I20240426 13:16:16.035205  9316 catalog_manager.cc:5484] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5606172f13e44bd1a9372805a8e67d8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:16.035921  9500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 3 candidate_status { last_received { term: 1 index: 3043 } } ignore_live_leader: false dest_uuid: "469ce5ad20d6420da45e9159560fbe64"
I20240426 13:16:16.035965  9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "469ce5ad20d6420da45e9159560fbe64" candidate_term: 2 candidate_status { last_received { term: 1 index: 3043 } } ignore_live_leader: false dest_uuid: "5606172f13e44bd1a9372805a8e67d8d"
I20240426 13:16:16.036051  9500 raft_consensus.cc:2379] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a08072d49eb746bda23d4045f3b6de1f in current term 3: Already voted for candidate 5606172f13e44bd1a9372805a8e67d8d in this term.
I20240426 13:16:16.036307  9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "469ce5ad20d6420da45e9159560fbe64" candidate_term: 2 candidate_status { last_received { term: 1 index: 3043 } } ignore_live_leader: false dest_uuid: "a08072d49eb746bda23d4045f3b6de1f"
I20240426 13:16:16.036422  9575 raft_consensus.cc:2354] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 469ce5ad20d6420da45e9159560fbe64 for earlier term 2. Current term is 3.
I20240426 13:16:16.036556  9540 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a08072d49eb746bda23d4045f3b6de1f; no voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d
I20240426 13:16:16.036691  9467 leader_election.cc:398] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 2 election: Vote denied by peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375) with higher term. Message: Invalid argument: T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 469ce5ad20d6420da45e9159560fbe64 for earlier term 2. Current term is 3.
I20240426 13:16:16.036737  9467 leader_election.cc:401] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20240426 13:16:16.036911  9850 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375) with higher term. Message: Invalid argument: T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 469ce5ad20d6420da45e9159560fbe64 for earlier term 2. Current term is 3.
I20240426 13:16:16.037137  9680 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240426 13:16:16.037274  9575 raft_consensus.cc:1261] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Refusing update from remote peer 5606172f13e44bd1a9372805a8e67d8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3043. Preceding OpId from leader: term: 3 index: 3045. (index mismatch)
I20240426 13:16:16.037532  9500 raft_consensus.cc:1261] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 3 FOLLOWER]: Refusing update from remote peer 5606172f13e44bd1a9372805a8e67d8d: Log matching property violated. Preceding OpId in replica: term: 1 index: 3043. Preceding OpId from leader: term: 3 index: 3045. (index mismatch)
I20240426 13:16:16.037572  9623 consensus_queue.cc:1035] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3044, Last known committed idx: 3043, Time since last communication: 0.000s
I20240426 13:16:16.037849  9623 consensus_queue.cc:1035] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3044, Last known committed idx: 3043, Time since last communication: 0.000s
I20240426 13:16:16.070407  9850 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5606172f13e44bd1a9372805a8e67d8d)
I20240426 13:16:16.070470  9850 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 3 FOLLOWER]: Advancing to term 4
I20240426 13:16:16.071244  9850 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.071570  9850 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 4 election: Requested vote from peers 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
I20240426 13:16:16.071787  9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "469ce5ad20d6420da45e9159560fbe64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3045 } } ignore_live_leader: false dest_uuid: "5606172f13e44bd1a9372805a8e67d8d"
I20240426 13:16:16.071823  9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "469ce5ad20d6420da45e9159560fbe64" candidate_term: 4 candidate_status { last_received { term: 3 index: 3045 } } ignore_live_leader: false dest_uuid: "a08072d49eb746bda23d4045f3b6de1f"
I20240426 13:16:16.072083  9467 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 469ce5ad20d6420da45e9159560fbe64; no voters: 5606172f13e44bd1a9372805a8e67d8d, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:16.072252  9850 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240426 13:16:16.075492  9500 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 4 FOLLOWER]: Rejecting Update request from peer 5606172f13e44bd1a9372805a8e67d8d for earlier term 3. Current term is 4. Ops: []
I20240426 13:16:16.075774  9664 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 }, Status: INVALID_TERM, Last received: 3.3045, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
W20240426 13:16:16.081849  9674 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 3 is higher.
I20240426 13:16:16.092458  9680 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5606172f13e44bd1a9372805a8e67d8d)
I20240426 13:16:16.092540  9680 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 3 FOLLOWER]: Advancing to term 4
I20240426 13:16:16.093549  9680 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.094022  9680 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 4 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405)
I20240426 13:16:16.094331  9500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3045 } } ignore_live_leader: false dest_uuid: "469ce5ad20d6420da45e9159560fbe64"
I20240426 13:16:16.094403  9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3045 } } ignore_live_leader: false dest_uuid: "5606172f13e44bd1a9372805a8e67d8d"
I20240426 13:16:16.094470  9500 raft_consensus.cc:2379] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a08072d49eb746bda23d4045f3b6de1f in current term 4: Already voted for candidate 469ce5ad20d6420da45e9159560fbe64 in this term.
I20240426 13:16:16.094767  9540 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a08072d49eb746bda23d4045f3b6de1f; no voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d
I20240426 13:16:16.094808  9575 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 4 FOLLOWER]: Rejecting Update request from peer 5606172f13e44bd1a9372805a8e67d8d for earlier term 3. Current term is 4. Ops: []
I20240426 13:16:16.094907  9680 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20240426 13:16:16.095098  9664 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: INVALID_TERM, Last received: 3.3045, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
I20240426 13:16:16.106671  9500 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 4 FOLLOWER]: Rejecting Update request from peer 5606172f13e44bd1a9372805a8e67d8d for earlier term 3. Current term is 4. Ops: []
I20240426 13:16:16.106986  9627 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 }, Status: INVALID_TERM, Last received: 3.3045, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
I20240426 13:16:16.108777  9850 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:16.108855  9850 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 4 FOLLOWER]: Advancing to term 5
I20240426 13:16:16.109656  9850 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.110246  9850 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 5 election: Requested vote from peers 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
I20240426 13:16:16.110519  9575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "469ce5ad20d6420da45e9159560fbe64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3045 } } ignore_live_leader: false dest_uuid: "a08072d49eb746bda23d4045f3b6de1f"
I20240426 13:16:16.110519  9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "469ce5ad20d6420da45e9159560fbe64" candidate_term: 5 candidate_status { last_received { term: 3 index: 3045 } } ignore_live_leader: false dest_uuid: "5606172f13e44bd1a9372805a8e67d8d"
I20240426 13:16:16.110634  9575 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 4 FOLLOWER]: Advancing to term 5
I20240426 13:16:16.111366  9575 raft_consensus.cc:2454] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 469ce5ad20d6420da45e9159560fbe64 in term 5.
I20240426 13:16:16.111549  9467 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f; no voters: 5606172f13e44bd1a9372805a8e67d8d
I20240426 13:16:16.111692  9850 raft_consensus.cc:2781] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 5 FOLLOWER]: Leader election won for term 5
I20240426 13:16:16.111842  9850 raft_consensus.cc:686] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 5 LEADER]: Becoming Leader. State: Replica: 469ce5ad20d6420da45e9159560fbe64, State: Running, Role: LEADER
I20240426 13:16:16.111989  9850 consensus_queue.cc:237] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3045, Committed index: 3045, Last appended: 3.3045, Last appended by leader: 3045, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.113181  9320 catalog_manager.cc:5484] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 reported cstate change: term changed from 3 to 5, leader changed from 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129) to 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130). New cstate: current_term: 5 leader_uuid: "469ce5ad20d6420da45e9159560fbe64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:16.117261  9575 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 5 FOLLOWER]: Rejecting Update request from peer 5606172f13e44bd1a9372805a8e67d8d for earlier term 3. Current term is 5. Ops: [3.3046-3.3046]
I20240426 13:16:16.117445  9576 raft_consensus.cc:1261] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 5 FOLLOWER]: Refusing update from remote peer 469ce5ad20d6420da45e9159560fbe64: Log matching property violated. Preceding OpId in replica: term: 3 index: 3045. Preceding OpId from leader: term: 5 index: 3047. (index mismatch)
I20240426 13:16:16.117542  9500 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 5 LEADER]: Rejecting Update request from peer 5606172f13e44bd1a9372805a8e67d8d for earlier term 3. Current term is 5. Ops: [3.3046-3.3046]
I20240426 13:16:16.117568  9623 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: INVALID_TERM, Last received: 3.3045, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
I20240426 13:16:16.117846  9664 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 }, Status: INVALID_TERM, Last received: 3.3045, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
I20240426 13:16:16.117998  9664 consensus_queue.cc:1225] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [LEADER]: Peer 469ce5ad20d6420da45e9159560fbe64 log is divergent from this leader: its last log entry 5.3047 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3045
I20240426 13:16:16.118139  9425 raft_consensus.cc:3032] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 3 LEADER]: Stepping down as leader of term 3
I20240426 13:16:16.118216  9425 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5606172f13e44bd1a9372805a8e67d8d, State: Running, Role: LEADER
I20240426 13:16:16.118253  9871 consensus_queue.cc:1035] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
I20240426 13:16:16.118336  9425 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3045, Committed index: 3045, Last appended: 3.3046, Last appended by leader: 3046, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.118449  9425 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 3 FOLLOWER]: Advancing to term 5
I20240426 13:16:16.119235  9425 raft_consensus.cc:1261] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 5 FOLLOWER]: Refusing update from remote peer 469ce5ad20d6420da45e9159560fbe64: Log matching property violated. Preceding OpId in replica: term: 3 index: 3046. Preceding OpId from leader: term: 5 index: 3047. (index mismatch)
I20240426 13:16:16.119535  9850 consensus_queue.cc:1035] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3046, Last known committed idx: 3045, Time since last communication: 0.000s
I20240426 13:16:16.119640  9850 consensus_queue.cc:1225] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [LEADER]: Peer 5606172f13e44bd1a9372805a8e67d8d log is divergent from this leader: its last log entry 3.3046 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3045
I20240426 13:16:16.120141  9425 pending_rounds.cc:76] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Aborting all ops after (but not including) 3045
I20240426 13:16:16.120188  9425 pending_rounds.cc:97] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Aborting uncommitted WRITE_OP operation due to leader change: 3.3046
I20240426 13:16:16.125635  9636 mvcc.cc:205] Tried to move back new op lower bound from 7021106692574171136 to 7021106692555972608. Current Snapshot: MvccSnapshot[applied={T|T < 7021106692574171136}]
I20240426 13:16:16.158432  9680 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 469ce5ad20d6420da45e9159560fbe64)
I20240426 13:16:16.158504  9680 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 5 FOLLOWER]: Advancing to term 6
I20240426 13:16:16.159138  9680 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.159399  9680 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 6 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405)
I20240426 13:16:16.159628  9500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3048 } } ignore_live_leader: false dest_uuid: "469ce5ad20d6420da45e9159560fbe64"
I20240426 13:16:16.159785  9425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3048 } } ignore_live_leader: false dest_uuid: "5606172f13e44bd1a9372805a8e67d8d"
I20240426 13:16:16.160143  9541 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a08072d49eb746bda23d4045f3b6de1f; no voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d
I20240426 13:16:16.160293  9680 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240426 13:16:16.163762  9576 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 6 FOLLOWER]: Rejecting Update request from peer 469ce5ad20d6420da45e9159560fbe64 for earlier term 5. Current term is 6. Ops: []
I20240426 13:16:16.164038  9871 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: INVALID_TERM, Last received: 5.3048, Next index: 3049, Last known committed idx: 3048, Time since last communication: 0.000s
I20240426 13:16:16.182353   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:16:16.186111   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:16.186332   398 tablet_replica.cc:330] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: stopping tablet replica
I20240426 13:16:16.186445   398 raft_consensus.cc:2229] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 5 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:16.190874  9576 raft_consensus.cc:1226] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 6 FOLLOWER]: Rejecting Update request from peer 469ce5ad20d6420da45e9159560fbe64 for earlier term 5. Current term is 6. Ops: []
I20240426 13:16:16.191154  9871 consensus_queue.cc:1046] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 }, Status: INVALID_TERM, Last received: 5.3048, Next index: 3049, Last known committed idx: 3048, Time since last communication: 0.000s
I20240426 13:16:16.199062  9680 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:16.199124  9680 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 6 FOLLOWER]: Advancing to term 7
I20240426 13:16:16.199687  9680 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:16.199934  9680 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 7 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405)
I20240426 13:16:16.200093  9878 raft_consensus.cc:3032] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 5 LEADER]: Stepping down as leader of term 5
I20240426 13:16:16.200135  9500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90b65ca9cfe8400baaff49156ed2a3d2" candidate_uuid: "a08072d49eb746bda23d4045f3b6de1f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3048 } } ignore_live_leader: false dest_uuid: "469ce5ad20d6420da45e9159560fbe64"
W20240426 13:16:16.200240  9541 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20240426 13:16:16.200143  9878 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 469ce5ad20d6420da45e9159560fbe64, State: Running, Role: LEADER
I20240426 13:16:16.200614  9878 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3048, Committed index: 3048, Last appended: 5.3048, Last appended by leader: 3048, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
W20240426 13:16:16.200861  9541 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20240426 13:16:16.200930  9878 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 5 FOLLOWER]: Advancing to term 6
I20240426 13:16:16.200954  9541 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a08072d49eb746bda23d4045f3b6de1f; no voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d
I20240426 13:16:16.201083  9680 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20240426 13:16:16.207666  9664 raft_consensus.cc:659] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: failed to trigger leader election: Illegal state: RaftConsensus is not running: State = Stopping [suppressed 5 similar messages]
I20240426 13:16:16.215871   398 raft_consensus.cc:2258] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 5 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:16.232178   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:16:16.233629   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:16:16.236491   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:16.236604   398 tablet_replica.cc:330] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: stopping tablet replica
I20240426 13:16:16.236685   398 raft_consensus.cc:2229] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 6 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:16.236761   398 raft_consensus.cc:2258] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 6 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:16.253001   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:16:16.254310   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:16:16.258134   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:16.258270   398 tablet_replica.cc:330] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: stopping tablet replica
I20240426 13:16:16.258347   398 raft_consensus.cc:2229] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 7 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:16.258424   398 raft_consensus.cc:2258] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 7 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:16.264537   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:16:16.265869   398 master.cc:537] Master@127.0.99.190:35049 shutting down...
I20240426 13:16:16.268777   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:16:16.268872   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:16.268922   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: stopping tablet replica
I20240426 13:16:16.280670   398 master.cc:552] Master@127.0.99.190:35049 shutdown complete.
I20240426 13:16:16.282709   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:16.285442  9889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:16.286350  9887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:16.287205   398 server_base.cc:1034] running on GCE node
W20240426 13:16:16.287988  9886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:16.288161   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:16.288197   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:16.288216   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137376288216 us; error 0 us; skew 500 ppm
I20240426 13:16:16.289044   398 webserver.cc:460] Webserver started at http://127.0.99.190:36539/ using document root <none> and password file <none>
I20240426 13:16:16.289161   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:16.289193   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:16:16.289937   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:16.290746  9894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:16.290944   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:16.291005   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "5bd9e4fe7a574d059ee98e460d03b15e"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:16.291065   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:16.306408   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:16.306639   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:16.310220   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:35049
I20240426 13:16:16.318080  9946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:35049 every 8 connection(s)
I20240426 13:16:16.319080  9947 sys_catalog.cc:263] Verifying existing consensus state
I20240426 13:16:16.319604  9947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: Bootstrap starting.
I20240426 13:16:16.325176  9947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:16:16.325397  9947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: Bootstrap complete.
I20240426 13:16:16.325706  9947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER }
I20240426 13:16:16.325778  9947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bd9e4fe7a574d059ee98e460d03b15e, State: Initialized, Role: FOLLOWER
I20240426 13:16:16.325865  9947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER }
I20240426 13:16:16.325932  9947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:16:16.325980  9947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:16:16.326032  9947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:16.326614  9947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER }
I20240426 13:16:16.326709  9947 leader_election.cc:302] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bd9e4fe7a574d059ee98e460d03b15e; no voters: 
I20240426 13:16:16.326828  9947 leader_election.cc:288] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [CANDIDATE]: Term 2 election: Requested vote from peers 
I20240426 13:16:16.326926  9950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 2 FOLLOWER]: Leader election won for term 2
I20240426 13:16:16.327081  9950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 2 LEADER]: Becoming Leader. State: Replica: 5bd9e4fe7a574d059ee98e460d03b15e, State: Running, Role: LEADER
I20240426 13:16:16.327212  9947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:16:16.327207  9950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER }
I20240426 13:16:16.327561  9951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER } }
I20240426 13:16:16.327613  9952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bd9e4fe7a574d059ee98e460d03b15e. Latest consensus state: current_term: 2 leader_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bd9e4fe7a574d059ee98e460d03b15e" member_type: VOTER } }
I20240426 13:16:16.327735  9951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: This master's current role is: LEADER
I20240426 13:16:16.327757  9952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [sys.catalog]: This master's current role is: LEADER
I20240426 13:16:16.328127  9957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:16:16.329222  9957 catalog_manager.cc:669] Loaded metadata for table test-workload [id=de2465e79a614b82bfc0c2df1d1471c8]
I20240426 13:16:16.329942  9957 tablet_loader.cc:96] loaded metadata for tablet 90b65ca9cfe8400baaff49156ed2a3d2 (table test-workload [id=de2465e79a614b82bfc0c2df1d1471c8])
I20240426 13:16:16.330081  9957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:16:16.330305   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:16:16.330555  9957 catalog_manager.cc:1259] Loaded cluster ID: 7b0675963f064e5bbcf304642f0ca1c8
I20240426 13:16:16.330597  9957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:16:16.331750  9957 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:16:16.332213  9957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e: Loaded TSK: 0
I20240426 13:16:16.332590  9957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:16:16.333931   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:16.336365  9966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:16.337152  9967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:16.337926  9969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:16.338793   398 server_base.cc:1034] running on GCE node
I20240426 13:16:16.338908   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:16.338946   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:16.338968   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137376338968 us; error 0 us; skew 500 ppm
I20240426 13:16:16.339732   398 webserver.cc:460] Webserver started at http://127.0.99.129:34489/ using document root <none> and password file <none>
I20240426 13:16:16.339852   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:16.339886   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:16:16.340709   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:16.341364  9974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:16.341559   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:16:16.341619   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "5606172f13e44bd1a9372805a8e67d8d"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:16.341676   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:16.349102   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:16.349352   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:16.349562   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:16.350050  9981 ts_tablet_manager.cc:516] Loading tablet metadata (0/1 complete)
I20240426 13:16:16.350785   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:16:16.350826   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:16:16.350880   398 ts_tablet_manager.cc:574] Registering tablets (0/1 complete)
I20240426 13:16:16.351454   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:16:16.351490   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:16:16.351527  9981 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Bootstrap starting.
I20240426 13:16:16.356184   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:45215
I20240426 13:16:16.356199 10040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:45215 every 8 connection(s)
I20240426 13:16:16.356863   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:16.359189 10045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:16.359817 10048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:16.360633 10046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:16.361122 10041 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35049
I20240426 13:16:16.361207 10041 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:16.361393 10041 heartbeater.cc:511] Master 127.0.99.190:35049 requested a full tablet report, sending...
I20240426 13:16:16.361464   398 server_base.cc:1034] running on GCE node
I20240426 13:16:16.361600   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:16.361629   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:16.361646   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137376361646 us; error 0 us; skew 500 ppm
I20240426 13:16:16.362479   398 webserver.cc:460] Webserver started at http://127.0.99.130:34819/ using document root <none> and password file <none>
I20240426 13:16:16.362458  9911 ts_manager.cc:194] Registered new tserver with Master: 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:45215)
I20240426 13:16:16.362627   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:16.362663   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:16:16.363695  9911 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34430
I20240426 13:16:16.364202   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:16:16.364956 10053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:16.365176   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:16.365264   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "469ce5ad20d6420da45e9159560fbe64"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:16.365324   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:16.384586   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:16.384843   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:16.385058   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:16.386607   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:16:16.386667   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:16:16.387353   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:16:16.387395   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:16:16.387446 10060 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Bootstrap starting.
I20240426 13:16:16.393193   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:35157
I20240426 13:16:16.393220 10118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:35157 every 8 connection(s)
I20240426 13:16:16.394017   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:16.396162 10122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:16.396987 10125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:16.401897 10119 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35049
I20240426 13:16:16.402012 10119 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:16.402217 10119 heartbeater.cc:511] Master 127.0.99.190:35049 requested a full tablet report, sending...
I20240426 13:16:16.402940  9911 ts_manager.cc:194] Registered new tserver with Master: 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35157)
W20240426 13:16:16.403034 10123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:16.403786  9911 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34432
I20240426 13:16:16.407496   398 server_base.cc:1034] running on GCE node
I20240426 13:16:16.407660   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:16.407703   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:16.407723   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137376407723 us; error 0 us; skew 500 ppm
I20240426 13:16:16.408630   398 webserver.cc:460] Webserver started at http://127.0.99.131:43437/ using document root <none> and password file <none>
I20240426 13:16:16.408769   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:16.408811   398 fs_manager.cc:362] Using existing metadata directory in first data directory
I20240426 13:16:16.409874   398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:16.410732 10130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:16.410950   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:16.411013   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "a08072d49eb746bda23d4045f3b6de1f"
format_stamp: "Formatted at 2024-04-26 13:16:05 on dist-test-slave-2jsl"
I20240426 13:16:16.411074   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestFrequentElections.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:16.424913   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:16.425290   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:16.425576   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:16.427103   398 ts_tablet_manager.cc:559] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20240426 13:16:16.427163   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.001s	user 0.000s	sys 0.000s
I20240426 13:16:16.428134   398 ts_tablet_manager.cc:590] Registered 1 tablets
I20240426 13:16:16.428189   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:16.428326 10137 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Bootstrap starting.
I20240426 13:16:16.434083   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:46149
I20240426 13:16:16.439949 10195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:46149 every 8 connection(s)
I20240426 13:16:16.444840 10196 heartbeater.cc:347] Connected to a master server at 127.0.99.190:35049
I20240426 13:16:16.444931 10196 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:16.445119 10196 heartbeater.cc:511] Master 127.0.99.190:35049 requested a full tablet report, sending...
I20240426 13:16:16.445756  9911 ts_manager.cc:194] Registered new tserver with Master: a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:46149)
I20240426 13:16:16.446588  9911 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34444
I20240426 13:16:16.446722   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.012213708s
I20240426 13:16:17.364573 10041 heartbeater.cc:503] Master 127.0.99.190:35049 was elected leader, sending a full tablet report...
I20240426 13:16:17.404650 10119 heartbeater.cc:503] Master 127.0.99.190:35049 was elected leader, sending a full tablet report...
I20240426 13:16:17.447407 10196 heartbeater.cc:503] Master 127.0.99.190:35049 was elected leader, sending a full tablet report...
I20240426 13:16:20.572521  9981 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Bootstrap replayed 1/1 log segments. Stats: ops{read=3049 overwritten=1 applied=3048 ignored=0} inserts{seen=152100 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:16:20.572909  9981 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Bootstrap complete.
I20240426 13:16:20.574241  9981 ts_tablet_manager.cc:1367] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Time spent bootstrapping tablet: real 4.223s	user 4.102s	sys 0.120s
I20240426 13:16:20.574617  9981 raft_consensus.cc:348] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:20.574745  9981 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5606172f13e44bd1a9372805a8e67d8d, State: Initialized, Role: FOLLOWER
I20240426 13:16:20.574904  9981 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3048, Last appended: 5.3048, Last appended by leader: 3048, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:20.575232  9981 ts_tablet_manager.cc:1398] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:20.611470 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:20.611572 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 5 FOLLOWER]: Advancing to term 6
I20240426 13:16:20.612607 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:20.613058 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 6 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:20.614251  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
W20240426 13:16:20.614336  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
I20240426 13:16:20.614421  9978 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:20.614578 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240426 13:16:20.646270 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:20.646356 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 6 FOLLOWER]: Advancing to term 7
I20240426 13:16:20.647316 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:20.647670 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 7 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:20.648793  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
W20240426 13:16:20.648926  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
I20240426 13:16:20.648994  9976 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:20.649211 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240426 13:16:20.681798 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:20.681874 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 7 FOLLOWER]: Advancing to term 8
I20240426 13:16:20.682668 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:20.682962 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 8 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:20.683844  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
W20240426 13:16:20.683949  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
I20240426 13:16:20.684005  9978 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:20.684168 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240426 13:16:20.739974 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:20.740051 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 8 FOLLOWER]: Advancing to term 9
I20240426 13:16:20.740824 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:20.741148 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 9 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:20.741959  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
W20240426 13:16:20.742323  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
I20240426 13:16:20.742389  9978 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:20.742559 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240426 13:16:20.872197 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:20.872289 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 9 FOLLOWER]: Advancing to term 10
I20240426 13:16:20.873090 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:20.873576 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 10 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:20.874418  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
W20240426 13:16:20.874481  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
I20240426 13:16:20.874542  9978 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:20.874698 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240426 13:16:20.944808 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:20.944896 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 10 FOLLOWER]: Advancing to term 11
I20240426 13:16:20.945732 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:20.946267 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 11 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:20.947073  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
W20240426 13:16:20.947196  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
I20240426 13:16:20.947268  9976 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:20.947419 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20240426 13:16:21.261369 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:21.261482 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 11 FOLLOWER]: Advancing to term 12
I20240426 13:16:21.262506 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:21.263203 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 12 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:21.263447  9976 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111) [suppressed 12 similar messages]
W20240426 13:16:21.264175  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
W20240426 13:16:21.264353  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
I20240426 13:16:21.264420  9976 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:21.264539 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20240426 13:16:21.390365 10060 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Bootstrap replaying log segment 1/1 (4.29M/5.05M this segment, stats: ops{read=2584 overwritten=0 applied=2578 ignored=0} inserts{seen=128850 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20240426 13:16:21.431182 10137 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Bootstrap replaying log segment 1/1 (4.39M/5.17M this segment, stats: ops{read=2584 overwritten=0 applied=2584 ignored=0} inserts{seen=129150 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20240426 13:16:21.548992 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:21.549106 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 12 FOLLOWER]: Advancing to term 13
I20240426 13:16:21.550196 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:21.550731 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 13 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:21.551729  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
W20240426 13:16:21.551806  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
I20240426 13:16:21.551879  9978 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:21.552003 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20240426 13:16:21.626598 10200 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:21.626696 10200 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 13 FOLLOWER]: Advancing to term 14
I20240426 13:16:21.627760 10200 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:21.628165 10200 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 14 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:21.629063  9978 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
W20240426 13:16:21.629783  9976 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
I20240426 13:16:21.629859  9976 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5606172f13e44bd1a9372805a8e67d8d; no voters: 469ce5ad20d6420da45e9159560fbe64, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:21.630039 10200 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20240426 13:16:22.050000 10060 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Bootstrap replayed 1/1 log segments. Stats: ops{read=3048 overwritten=0 applied=3048 ignored=0} inserts{seen=152100 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:16:22.050319 10060 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Bootstrap complete.
I20240426 13:16:22.051395 10060 ts_tablet_manager.cc:1367] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Time spent bootstrapping tablet: real 5.664s	user 5.546s	sys 0.116s
I20240426 13:16:22.051716 10060 raft_consensus.cc:348] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:22.051810 10060 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 469ce5ad20d6420da45e9159560fbe64, State: Initialized, Role: FOLLOWER
I20240426 13:16:22.051932 10060 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3048, Last appended: 5.3048, Last appended by leader: 3048, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:22.052229 10060 ts_tablet_manager.cc:1398] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:16:22.069805 10137 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3048 overwritten=0 applied=3048 ignored=0} inserts{seen=152100 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20240426 13:16:22.070103 10137 tablet_bootstrap.cc:492] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Bootstrap complete.
I20240426 13:16:22.071193 10137 ts_tablet_manager.cc:1367] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Time spent bootstrapping tablet: real 5.643s	user 5.533s	sys 0.100s
I20240426 13:16:22.071473 10137 raft_consensus.cc:348] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:22.071557 10137 raft_consensus.cc:729] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: a08072d49eb746bda23d4045f3b6de1f, State: Initialized, Role: FOLLOWER
I20240426 13:16:22.071651 10137 consensus_queue.cc:260] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3048, Last appended: 5.3048, Last appended by leader: 3048, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:22.071918 10137 ts_tablet_manager.cc:1398] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:16:22.072239   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:16:22.075560   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:22.075688   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:16:22.075798   398 raft_consensus.cc:2229] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 14 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:22.075877   398 raft_consensus.cc:2258] T 90b65ca9cfe8400baaff49156ed2a3d2 P 5606172f13e44bd1a9372805a8e67d8d [term 14 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:22.079481 10204 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:22.079555 10204 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 6 FOLLOWER]: Advancing to term 7
I20240426 13:16:22.080556 10204 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:22.080883 10204 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 7 election: Requested vote from peers 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405), a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375)
W20240426 13:16:22.082028 10056 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405): Network error: Client connection negotiation failed: client connection to 127.0.99.129:35405: connect: Connection refused (error 111)
W20240426 13:16:22.082129 10057 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a08072d49eb746bda23d4045f3b6de1f (127.0.99.131:35375): Network error: Client connection negotiation failed: client connection to 127.0.99.131:35375: connect: Connection refused (error 111)
I20240426 13:16:22.082216 10057 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 469ce5ad20d6420da45e9159560fbe64; no voters: 5606172f13e44bd1a9372805a8e67d8d, a08072d49eb746bda23d4045f3b6de1f
I20240426 13:16:22.082340 10204 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20240426 13:16:22.092419   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:16:22.093909   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:16:22.096959   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:22.097139   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:16:22.097218   398 raft_consensus.cc:2229] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 7 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:22.097302   398 raft_consensus.cc:2258] T 90b65ca9cfe8400baaff49156ed2a3d2 P 469ce5ad20d6420da45e9159560fbe64 [term 7 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:22.101716 10205 raft_consensus.cc:482] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:22.101811 10205 raft_consensus.cc:3037] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 7 FOLLOWER]: Advancing to term 8
I20240426 13:16:22.103116 10205 raft_consensus.cc:504] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "469ce5ad20d6420da45e9159560fbe64" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 35953 } } peers { permanent_uuid: "5606172f13e44bd1a9372805a8e67d8d" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 35405 } } peers { permanent_uuid: "a08072d49eb746bda23d4045f3b6de1f" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 35375 } }
I20240426 13:16:22.103593 10205 leader_election.cc:288] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 8 election: Requested vote from peers 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953), 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405)
W20240426 13:16:22.104533 10132 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 469ce5ad20d6420da45e9159560fbe64 (127.0.99.130:35953): Network error: Client connection negotiation failed: client connection to 127.0.99.130:35953: connect: Connection refused (error 111)
W20240426 13:16:22.104658 10133 leader_election.cc:334] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 5606172f13e44bd1a9372805a8e67d8d (127.0.99.129:35405): Network error: Client connection negotiation failed: client connection to 127.0.99.129:35405: connect: Connection refused (error 111)
I20240426 13:16:22.104738 10133 leader_election.cc:302] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a08072d49eb746bda23d4045f3b6de1f; no voters: 469ce5ad20d6420da45e9159560fbe64, 5606172f13e44bd1a9372805a8e67d8d
I20240426 13:16:22.104869 10205 raft_consensus.cc:2726] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240426 13:16:22.115971   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:16:22.117357   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:16:22.121191   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:22.121340   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:16:22.121423   398 raft_consensus.cc:2229] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 8 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:22.121507   398 raft_consensus.cc:2258] T 90b65ca9cfe8400baaff49156ed2a3d2 P a08072d49eb746bda23d4045f3b6de1f [term 8 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:22.136914   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:16:22.138345   398 master.cc:537] Master@127.0.99.190:35049 shutting down...
I20240426 13:16:22.140931   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 2 LEADER]: Raft consensus shutting down.
I20240426 13:16:22.141043   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5bd9e4fe7a574d059ee98e460d03b15e [term 2 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:22.141098   398 tablet_replica.cc:330] stopping tablet replica
I20240426 13:16:22.152611   398 master.cc:552] Master@127.0.99.190:35049 shutdown complete.
[       OK ] TxnParticipantElectionStormITest.TestFrequentElections (16440 ms)
[ RUN      ] TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm
I20240426 13:16:22.160720   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:42111
I20240426 13:16:22.160997   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:22.163208 10212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:22.164119 10213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:22.164944 10215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:22.165737   398 server_base.cc:1034] running on GCE node
I20240426 13:16:22.165846   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:22.165877   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:22.165895   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137382165896 us; error 0 us; skew 500 ppm
I20240426 13:16:22.177594   398 webserver.cc:460] Webserver started at http://127.0.99.190:45129/ using document root <none> and password file <none>
I20240426 13:16:22.177716   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:22.177757   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:22.177819   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:22.178177   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "cfe79240f4974fd399668ae5ea8234d5"
format_stamp: "Formatted at 2024-04-26 13:16:22 on dist-test-slave-2jsl"
I20240426 13:16:22.179464   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:22.180188 10220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.180393   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:16:22.180449   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "cfe79240f4974fd399668ae5ea8234d5"
format_stamp: "Formatted at 2024-04-26 13:16:22 on dist-test-slave-2jsl"
I20240426 13:16:22.180505   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:22.195276   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:22.195501   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:22.199152   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:42111
I20240426 13:16:22.201905 10272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:42111 every 8 connection(s)
I20240426 13:16:22.202221 10273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:22.203557 10273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5: Bootstrap starting.
I20240426 13:16:22.204113 10273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:22.204850 10273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5: No bootstrap required, opened a new log
I20240426 13:16:22.205108 10273 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe79240f4974fd399668ae5ea8234d5" member_type: VOTER }
I20240426 13:16:22.205173 10273 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:22.205197 10273 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe79240f4974fd399668ae5ea8234d5, State: Initialized, Role: FOLLOWER
I20240426 13:16:22.205272 10273 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe79240f4974fd399668ae5ea8234d5" member_type: VOTER }
I20240426 13:16:22.205315 10273 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:16:22.205338 10273 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:16:22.205368 10273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:22.205889 10273 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe79240f4974fd399668ae5ea8234d5" member_type: VOTER }
I20240426 13:16:22.206007 10273 leader_election.cc:302] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfe79240f4974fd399668ae5ea8234d5; no voters: 
I20240426 13:16:22.206127 10273 leader_election.cc:288] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:16:22.206225 10276 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:16:22.206389 10276 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 1 LEADER]: Becoming Leader. State: Replica: cfe79240f4974fd399668ae5ea8234d5, State: Running, Role: LEADER
I20240426 13:16:22.206483 10273 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:16:22.206506 10276 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe79240f4974fd399668ae5ea8234d5" member_type: VOTER }
I20240426 13:16:22.206992 10278 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfe79240f4974fd399668ae5ea8234d5. Latest consensus state: current_term: 1 leader_uuid: "cfe79240f4974fd399668ae5ea8234d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe79240f4974fd399668ae5ea8234d5" member_type: VOTER } }
I20240426 13:16:22.206980 10277 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfe79240f4974fd399668ae5ea8234d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfe79240f4974fd399668ae5ea8234d5" member_type: VOTER } }
I20240426 13:16:22.207108 10278 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [sys.catalog]: This master's current role is: LEADER
I20240426 13:16:22.207182 10277 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [sys.catalog]: This master's current role is: LEADER
I20240426 13:16:22.207634 10282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:16:22.208243 10282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:16:22.208364   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:16:22.209658 10282 catalog_manager.cc:1347] Generated new cluster ID: be03cb729bb849768d6485e4d771d356
I20240426 13:16:22.209709 10282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:16:22.219187 10282 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:16:22.219626 10282 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:16:22.227970 10282 catalog_manager.cc:5857] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5: Generated new TSK 0
I20240426 13:16:22.228130 10282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:16:22.240303   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:22.242561 10294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:22.243336 10295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:22.244174 10297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:22.244916   398 server_base.cc:1034] running on GCE node
I20240426 13:16:22.245030   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:22.245059   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:22.245079   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137382245077 us; error 0 us; skew 500 ppm
I20240426 13:16:22.245733   398 webserver.cc:460] Webserver started at http://127.0.99.129:38439/ using document root <none> and password file <none>
I20240426 13:16:22.245846   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:22.245883   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:22.245957   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:22.246444   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "5c038d4dde124711bb12ef6d54c5b72b"
format_stamp: "Formatted at 2024-04-26 13:16:22 on dist-test-slave-2jsl"
I20240426 13:16:22.248076   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:16:22.248852 10302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.249035   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.000s	sys 0.001s
I20240426 13:16:22.249095   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "5c038d4dde124711bb12ef6d54c5b72b"
format_stamp: "Formatted at 2024-04-26 13:16:22 on dist-test-slave-2jsl"
I20240426 13:16:22.249154   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:22.271764   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:22.271996   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:22.272220   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:22.272567   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:22.272600   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.272635   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:22.272671   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.276265   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:37747
I20240426 13:16:22.277101 10365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:37747 every 8 connection(s)
I20240426 13:16:22.277374   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:22.279589 10370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:22.280263 10373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:22.280936 10371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:22.281388 10366 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42111
I20240426 13:16:22.281462 10366 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:22.281639   398 server_base.cc:1034] running on GCE node
I20240426 13:16:22.281663 10366 heartbeater.cc:511] Master 127.0.99.190:42111 requested a full tablet report, sending...
I20240426 13:16:22.281766   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:22.281805   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:22.281826   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137382281825 us; error 0 us; skew 500 ppm
I20240426 13:16:22.282298 10237 ts_manager.cc:194] Registered new tserver with Master: 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:22.282653   398 webserver.cc:460] Webserver started at http://127.0.99.130:46269/ using document root <none> and password file <none>
I20240426 13:16:22.282775   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:22.282814   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:22.282876   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:22.283084 10237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38358
I20240426 13:16:22.283174   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "b67b075c1241471eb4784ac2e1087916"
format_stamp: "Formatted at 2024-04-26 13:16:22 on dist-test-slave-2jsl"
I20240426 13:16:22.284384   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.001s
I20240426 13:16:22.285090 10378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.285264   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:16:22.285319   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "b67b075c1241471eb4784ac2e1087916"
format_stamp: "Formatted at 2024-04-26 13:16:22 on dist-test-slave-2jsl"
I20240426 13:16:22.285373   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:22.315528   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:22.315764   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:22.315980   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:22.316314   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:22.316349   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.316386   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:22.316414   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.320050   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:38087
I20240426 13:16:22.320093 10441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:38087 every 8 connection(s)
I20240426 13:16:22.320715   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:22.322645 10445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:22.323321 10446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:22.324103 10448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:22.324223 10442 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42111
I20240426 13:16:22.324294 10442 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:22.324452 10442 heartbeater.cc:511] Master 127.0.99.190:42111 requested a full tablet report, sending...
I20240426 13:16:22.324782   398 server_base.cc:1034] running on GCE node
I20240426 13:16:22.324935   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:22.324965   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:22.324983   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137382324983 us; error 0 us; skew 500 ppm
I20240426 13:16:22.325006 10237 ts_manager.cc:194] Registered new tserver with Master: b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087)
I20240426 13:16:22.325798   398 webserver.cc:460] Webserver started at http://127.0.99.131:41461/ using document root <none> and password file <none>
I20240426 13:16:22.325914   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:22.325956   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:22.325955 10237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38364
I20240426 13:16:22.326062   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:22.326491   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
format_stamp: "Formatted at 2024-04-26 13:16:22 on dist-test-slave-2jsl"
I20240426 13:16:22.327721   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:22.328474 10453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.328660   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:16:22.328730   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
format_stamp: "Formatted at 2024-04-26 13:16:22 on dist-test-slave-2jsl"
I20240426 13:16:22.328789   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:22.332695   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:22.332901   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:22.333104   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:22.333416   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:22.333448   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.333484   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:22.333508   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:22.337548   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:43429
I20240426 13:16:22.337595 10516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:43429 every 8 connection(s)
I20240426 13:16:22.341055 10517 heartbeater.cc:347] Connected to a master server at 127.0.99.190:42111
I20240426 13:16:22.341116 10517 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:22.341256 10517 heartbeater.cc:511] Master 127.0.99.190:42111 requested a full tablet report, sending...
I20240426 13:16:22.341678 10237 ts_manager.cc:194] Registered new tserver with Master: e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:22.342113 10237 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38372
I20240426 13:16:22.342406   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00449506s
I20240426 13:16:22.342830   398 test_util.cc:268] Using random seed: -1245359954
I20240426 13:16:22.348668 10237 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38378:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:16:22.349027 10237 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:16:22.356458 10406 tablet_service.cc:1467] Processing CreateTablet for tablet ad1902292fbd419ebbf379c83ee3d9f6 (DEFAULT_TABLE table=test-workload [id=0a3c2bc9cf604fe982d40ae94328ed3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:22.356737 10406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1902292fbd419ebbf379c83ee3d9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:22.357019 10330 tablet_service.cc:1467] Processing CreateTablet for tablet ad1902292fbd419ebbf379c83ee3d9f6 (DEFAULT_TABLE table=test-workload [id=0a3c2bc9cf604fe982d40ae94328ed3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:22.357201 10330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1902292fbd419ebbf379c83ee3d9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:22.357846 10481 tablet_service.cc:1467] Processing CreateTablet for tablet ad1902292fbd419ebbf379c83ee3d9f6 (DEFAULT_TABLE table=test-workload [id=0a3c2bc9cf604fe982d40ae94328ed3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:22.358048 10481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad1902292fbd419ebbf379c83ee3d9f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:22.359231 10533 tablet_bootstrap.cc:492] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b: Bootstrap starting.
I20240426 13:16:22.359859 10532 tablet_bootstrap.cc:492] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916: Bootstrap starting.
I20240426 13:16:22.360131 10533 tablet_bootstrap.cc:654] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:22.360642 10532 tablet_bootstrap.cc:654] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:22.361079 10533 tablet_bootstrap.cc:492] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b: No bootstrap required, opened a new log
I20240426 13:16:22.361163 10533 ts_tablet_manager.cc:1367] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:16:22.361218 10534 tablet_bootstrap.cc:492] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb: Bootstrap starting.
I20240426 13:16:22.361496 10532 tablet_bootstrap.cc:492] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916: No bootstrap required, opened a new log
I20240426 13:16:22.361510 10533 raft_consensus.cc:348] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.361578 10532 ts_tablet_manager.cc:1367] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:16:22.361630 10533 raft_consensus.cc:374] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:22.361665 10533 raft_consensus.cc:729] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c038d4dde124711bb12ef6d54c5b72b, State: Initialized, Role: FOLLOWER
I20240426 13:16:22.361799 10533 consensus_queue.cc:260] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.362007 10534 tablet_bootstrap.cc:654] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:22.361953 10532 raft_consensus.cc:348] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.362057 10532 raft_consensus.cc:374] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:22.362102 10532 raft_consensus.cc:729] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b67b075c1241471eb4784ac2e1087916, State: Initialized, Role: FOLLOWER
I20240426 13:16:22.362167 10533 ts_tablet_manager.cc:1398] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:22.362242 10532 consensus_queue.cc:260] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.362296 10366 heartbeater.cc:503] Master 127.0.99.190:42111 was elected leader, sending a full tablet report...
I20240426 13:16:22.362545 10532 ts_tablet_manager.cc:1398] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:22.362695 10442 heartbeater.cc:503] Master 127.0.99.190:42111 was elected leader, sending a full tablet report...
I20240426 13:16:22.362910 10537 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:16:22.362985 10537 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.363633 10537 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:22.364153 10534 tablet_bootstrap.cc:492] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb: No bootstrap required, opened a new log
I20240426 13:16:22.364241 10534 ts_tablet_manager.cc:1367] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:16:22.364640 10534 raft_consensus.cc:348] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.364742 10534 raft_consensus.cc:374] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:22.364776 10534 raft_consensus.cc:729] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e616a8ecb24e4625bc45ec76a5bd1bfb, State: Initialized, Role: FOLLOWER
I20240426 13:16:22.364934 10534 consensus_queue.cc:260] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.365281 10534 ts_tablet_manager.cc:1398] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:22.365859 10517 heartbeater.cc:503] Master 127.0.99.190:42111 was elected leader, sending a full tablet report...
I20240426 13:16:22.366958 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" is_pre_election: true
I20240426 13:16:22.367058 10490 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c038d4dde124711bb12ef6d54c5b72b in term 0.
I20240426 13:16:22.367655 10306 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b, e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 
I20240426 13:16:22.367825 10537 raft_consensus.cc:2781] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:16:22.367878 10537 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:22.367918 10537 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:22.368582 10537 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.368861 10537 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 1 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:22.369165 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:22.369254 10490 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:22.369469 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:22.369469 10416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916" is_pre_election: true
I20240426 13:16:22.369601 10415 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:22.370045 10490 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c038d4dde124711bb12ef6d54c5b72b in term 1.
I20240426 13:16:22.370271 10306 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b, e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 
I20240426 13:16:22.370440 10537 raft_consensus.cc:2781] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:16:22.370484 10415 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c038d4dde124711bb12ef6d54c5b72b in term 1.
I20240426 13:16:22.370597 10537 raft_consensus.cc:686] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 1 LEADER]: Becoming Leader. State: Replica: 5c038d4dde124711bb12ef6d54c5b72b, State: Running, Role: LEADER
I20240426 13:16:22.370743 10537 consensus_queue.cc:237] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:22.371943 10237 catalog_manager.cc:5484] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129). New cstate: current_term: 1 leader_uuid: "5c038d4dde124711bb12ef6d54c5b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:22.397233 10415 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 1 FOLLOWER]: Refusing update from remote peer 5c038d4dde124711bb12ef6d54c5b72b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:16:22.397249 10490 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 1 FOLLOWER]: Refusing update from remote peer 5c038d4dde124711bb12ef6d54c5b72b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:16:22.397531 10542 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:16:22.397624 10537 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:16:32.440160 10599 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c038d4dde124711bb12ef6d54c5b72b)
I20240426 13:16:32.440235 10599 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:32.440929 10599 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.441236 10599 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 2 election: Requested vote from peers 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.444154 10415 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 2 FOLLOWER]: Rejecting Update request from peer 5c038d4dde124711bb12ef6d54c5b72b for earlier term 1. Current term is 2. Ops: []
I20240426 13:16:32.444511 10567 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 }, Status: INVALID_TERM, Last received: 1.3001, Next index: 3002, Last known committed idx: 3001, Time since last communication: 0.000s
I20240426 13:16:32.444885 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 2 candidate_status { last_received { term: 1 index: 3001 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.445078 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 2 candidate_status { last_received { term: 1 index: 3001 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.445307 10380 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b67b075c1241471eb4784ac2e1087916; no voters: 5c038d4dde124711bb12ef6d54c5b72b, e616a8ecb24e4625bc45ec76a5bd1bfb
I20240426 13:16:32.445443 10599 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20240426 13:16:32.460985 10415 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 2 FOLLOWER]: Rejecting Update request from peer 5c038d4dde124711bb12ef6d54c5b72b for earlier term 1. Current term is 2. Ops: [1.3002-1.3002]
I20240426 13:16:32.461279 10585 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 }, Status: INVALID_TERM, Last received: 1.3001, Next index: 3002, Last known committed idx: 3001, Time since last communication: 0.000s
I20240426 13:16:32.465821 10567 raft_consensus.cc:3032] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 1 LEADER]: Stepping down as leader of term 1
I20240426 13:16:32.465873 10567 raft_consensus.cc:729] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5c038d4dde124711bb12ef6d54c5b72b, State: Running, Role: LEADER
I20240426 13:16:32.465996 10567 consensus_queue.cc:260] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3002, Committed index: 3002, Last appended: 1.3002, Last appended by leader: 3002, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.466117 10567 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:32.483043 10599 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.483105 10599 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:16:32.483716 10599 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.484019 10599 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 3 election: Requested vote from peers 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.484246 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 3 candidate_status { last_received { term: 1 index: 3001 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.484333 10340 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 2 FOLLOWER]: Advancing to term 3
I20240426 13:16:32.484295 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 3 candidate_status { last_received { term: 1 index: 3001 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.485118 10340 raft_consensus.cc:2396] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b67b075c1241471eb4784ac2e1087916 for term 3 because replica has last-logged OpId of term: 1 index: 3002, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3001.
I20240426 13:16:32.485381 10380 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b67b075c1241471eb4784ac2e1087916; no voters: 5c038d4dde124711bb12ef6d54c5b72b, e616a8ecb24e4625bc45ec76a5bd1bfb
I20240426 13:16:32.485549 10599 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20240426 13:16:32.491467 10585 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.491534 10585 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 3 FOLLOWER]: Advancing to term 4
I20240426 13:16:32.492185 10585 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.492377 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5c038d4dde124711bb12ef6d54c5b72b)
I20240426 13:16:32.492436 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 1 FOLLOWER]: Advancing to term 2
I20240426 13:16:32.492496 10585 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 4 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.492751 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3002 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.492794 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 4 candidate_status { last_received { term: 1 index: 3002 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.492848 10415 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 3 FOLLOWER]: Advancing to term 4
I20240426 13:16:32.493268 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.493431 10491 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 2 FOLLOWER]: Advancing to term 4
I20240426 13:16:32.493484 10415 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c038d4dde124711bb12ef6d54c5b72b in term 4.
I20240426 13:16:32.493688 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 2 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:32.494072 10305 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916; no voters: 
I20240426 13:16:32.494235 10585 raft_consensus.cc:2781] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 4 FOLLOWER]: Leader election won for term 4
I20240426 13:16:32.494308 10585 raft_consensus.cc:686] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 4 LEADER]: Becoming Leader. State: Replica: 5c038d4dde124711bb12ef6d54c5b72b, State: Running, Role: LEADER
I20240426 13:16:32.494343 10491 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c038d4dde124711bb12ef6d54c5b72b in term 4.
I20240426 13:16:32.494442 10585 consensus_queue.cc:237] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3001, Committed index: 3001, Last appended: 1.3002, Last appended by leader: 3002, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.495751 10236 catalog_manager.cc:5484] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "5c038d4dde124711bb12ef6d54c5b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:32.497381 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3002 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.497493 10415 raft_consensus.cc:2354] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e616a8ecb24e4625bc45ec76a5bd1bfb for earlier term 2. Current term is 4.
I20240426 13:16:32.497599 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 3002 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.497798 10456 leader_election.cc:398] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 2 election: Vote denied by peer b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087) with higher term. Message: Invalid argument: T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e616a8ecb24e4625bc45ec76a5bd1bfb for earlier term 2. Current term is 4.
I20240426 13:16:32.497859 10456 leader_election.cc:401] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20240426 13:16:32.497992 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 4 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087) with higher term. Message: Invalid argument: T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e616a8ecb24e4625bc45ec76a5bd1bfb for earlier term 2. Current term is 4.
W20240426 13:16:32.515028 10585 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 4 is higher.
I20240426 13:16:32.522789 10415 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 4 FOLLOWER]: Refusing update from remote peer 5c038d4dde124711bb12ef6d54c5b72b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3001. Preceding OpId from leader: term: 4 index: 3003. (index mismatch)
I20240426 13:16:32.522893 10491 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 4 FOLLOWER]: Refusing update from remote peer 5c038d4dde124711bb12ef6d54c5b72b: Log matching property violated. Preceding OpId in replica: term: 1 index: 3002. Preceding OpId from leader: term: 4 index: 3003. (index mismatch)
I20240426 13:16:32.523102 10567 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3003, Last known committed idx: 3001, Time since last communication: 0.000s
I20240426 13:16:32.523227 10585 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3003, Last known committed idx: 3002, Time since last communication: 0.000s
I20240426 13:16:32.560858 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c038d4dde124711bb12ef6d54c5b72b)
I20240426 13:16:32.560920 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 4 FOLLOWER]: Advancing to term 5
I20240426 13:16:32.561553 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.561829 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 5 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:32.562080 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3005 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.562144 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 5 candidate_status { last_received { term: 4 index: 3005 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.562458 10455 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916
I20240426 13:16:32.562595 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20240426 13:16:32.562939 10491 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 5 FOLLOWER]: Rejecting Update request from peer 5c038d4dde124711bb12ef6d54c5b72b for earlier term 4. Current term is 5. Ops: []
I20240426 13:16:32.563231 10542 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 4.3005, Next index: 3006, Last known committed idx: 3005, Time since last communication: 0.000s
I20240426 13:16:32.582917 10491 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 5 FOLLOWER]: Rejecting Update request from peer 5c038d4dde124711bb12ef6d54c5b72b for earlier term 4. Current term is 5. Ops: [4.3006-4.3006]
I20240426 13:16:32.583328 10542 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 4.3005, Next index: 3006, Last known committed idx: 3005, Time since last communication: 0.000s
I20240426 13:16:32.598078 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.598141 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 5 FOLLOWER]: Advancing to term 6
I20240426 13:16:32.598739 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.599066 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 6 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:32.599334 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3005 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.599359 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 6 candidate_status { last_received { term: 4 index: 3005 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.599725 10455 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916
I20240426 13:16:32.599881 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20240426 13:16:32.605566 10585 raft_consensus.cc:3032] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 4 LEADER]: Stepping down as leader of term 4
I20240426 13:16:32.605618 10585 raft_consensus.cc:729] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5c038d4dde124711bb12ef6d54c5b72b, State: Running, Role: LEADER
I20240426 13:16:32.605737 10585 consensus_queue.cc:260] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3006, Committed index: 3006, Last appended: 4.3006, Last appended by leader: 3006, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.605865 10585 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 4 FOLLOWER]: Advancing to term 5
I20240426 13:16:32.614058 10599 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5c038d4dde124711bb12ef6d54c5b72b)
I20240426 13:16:32.614127 10599 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 4 FOLLOWER]: Advancing to term 5
I20240426 13:16:32.614727 10599 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.614996 10599 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 5 election: Requested vote from peers 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.615226 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 5 candidate_status { last_received { term: 4 index: 3006 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.615244 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 5 candidate_status { last_received { term: 4 index: 3006 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.615340 10491 raft_consensus.cc:2354] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b67b075c1241471eb4784ac2e1087916 for earlier term 5. Current term is 6.
I20240426 13:16:32.615631 10382 leader_election.cc:398] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 5 election: Vote denied by peer e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429) with higher term. Message: Invalid argument: T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b67b075c1241471eb4784ac2e1087916 for earlier term 5. Current term is 6.
I20240426 13:16:32.615675 10382 leader_election.cc:401] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20240426 13:16:32.615772 10599 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 5 FOLLOWER]: Advancing to term 6
I20240426 13:16:32.616019 10340 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b67b075c1241471eb4784ac2e1087916 in term 5.
I20240426 13:16:32.616415 10599 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429) with higher term. Message: Invalid argument: T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate b67b075c1241471eb4784ac2e1087916 for earlier term 5. Current term is 6.
I20240426 13:16:32.651432 10585 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.651516 10585 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 5 FOLLOWER]: Advancing to term 6
I20240426 13:16:32.651960 10599 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.652031 10599 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 6 FOLLOWER]: Advancing to term 7
I20240426 13:16:32.652186 10585 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.652500 10585 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 6 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.652763 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3006 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.652729 10599 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.652731 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 6 candidate_status { last_received { term: 4 index: 3006 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.652891 10491 raft_consensus.cc:2379] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c038d4dde124711bb12ef6d54c5b72b in current term 6: Already voted for candidate e616a8ecb24e4625bc45ec76a5bd1bfb in this term.
I20240426 13:16:32.652901 10415 raft_consensus.cc:2354] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c038d4dde124711bb12ef6d54c5b72b for earlier term 6. Current term is 7.
I20240426 13:16:32.653146 10599 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 7 election: Requested vote from peers 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.653318 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 7 candidate_status { last_received { term: 4 index: 3006 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.653323 10305 leader_election.cc:398] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 6 election: Vote denied by peer b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087) with higher term. Message: Invalid argument: T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c038d4dde124711bb12ef6d54c5b72b for earlier term 6. Current term is 7.
I20240426 13:16:32.653441 10305 leader_election.cc:401] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20240426 13:16:32.653455 10340 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 6 FOLLOWER]: Advancing to term 7
I20240426 13:16:32.653481 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 7 candidate_status { last_received { term: 4 index: 3006 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.653591 10491 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 6 FOLLOWER]: Advancing to term 7
I20240426 13:16:32.654110 10340 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b67b075c1241471eb4784ac2e1087916 in term 7.
I20240426 13:16:32.654201 10585 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087) with higher term. Message: Invalid argument: T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c038d4dde124711bb12ef6d54c5b72b for earlier term 6. Current term is 7.
I20240426 13:16:32.654300 10491 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b67b075c1241471eb4784ac2e1087916 in term 7.
I20240426 13:16:32.654357 10380 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916; no voters: 
I20240426 13:16:32.654573 10599 raft_consensus.cc:2781] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 FOLLOWER]: Leader election won for term 7
I20240426 13:16:32.654770 10599 raft_consensus.cc:686] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 LEADER]: Becoming Leader. State: Replica: b67b075c1241471eb4784ac2e1087916, State: Running, Role: LEADER
I20240426 13:16:32.654891 10599 consensus_queue.cc:237] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3006, Committed index: 3006, Last appended: 4.3006, Last appended by leader: 3006, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.655946 10236 catalog_manager.cc:5484] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 reported cstate change: term changed from 4 to 7, leader changed from 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129) to b67b075c1241471eb4784ac2e1087916 (127.0.99.130). New cstate: current_term: 7 leader_uuid: "b67b075c1241471eb4784ac2e1087916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:32.682052 10491 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 7 FOLLOWER]: Refusing update from remote peer b67b075c1241471eb4784ac2e1087916: Log matching property violated. Preceding OpId in replica: term: 4 index: 3005. Preceding OpId from leader: term: 7 index: 3007. (index mismatch)
I20240426 13:16:32.682343 10599 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3007, Last known committed idx: 3005, Time since last communication: 0.000s
I20240426 13:16:32.690177 10340 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 7 FOLLOWER]: Refusing update from remote peer b67b075c1241471eb4784ac2e1087916: Log matching property violated. Preceding OpId in replica: term: 4 index: 3006. Preceding OpId from leader: term: 7 index: 3007. (index mismatch)
I20240426 13:16:32.690491 10599 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3007, Last known committed idx: 3006, Time since last communication: 0.000s
W20240426 13:16:32.691936 10330 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:60998: Illegal state: replica 5c038d4dde124711bb12ef6d54c5b72b is not leader of this config: current role FOLLOWER
I20240426 13:16:32.730297 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 7 FOLLOWER]: Starting leader election (detected failure of leader b67b075c1241471eb4784ac2e1087916)
I20240426 13:16:32.730366 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 7 FOLLOWER]: Advancing to term 8
I20240426 13:16:32.730995 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.731062 10585 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 7 FOLLOWER]: Starting leader election (detected failure of leader b67b075c1241471eb4784ac2e1087916)
I20240426 13:16:32.731156 10491 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 8 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 8. Ops: []
I20240426 13:16:32.731328 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 8 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:32.731554 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 8 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.731341 10585 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 7 FOLLOWER]: Advancing to term 8
I20240426 13:16:32.731544 10614 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.731786 10339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 8 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.732074 10455 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916
I20240426 13:16:32.732241 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240426 13:16:32.732339 10585 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.732491 10340 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 8 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 8. Ops: []
I20240426 13:16:32.732646 10585 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 8 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.732857 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.732883 10614 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.732931 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 8 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.733050 10491 raft_consensus.cc:2379] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c038d4dde124711bb12ef6d54c5b72b in current term 8: Already voted for candidate e616a8ecb24e4625bc45ec76a5bd1bfb in this term.
I20240426 13:16:32.733232 10306 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b; no voters: b67b075c1241471eb4784ac2e1087916, e616a8ecb24e4625bc45ec76a5bd1bfb
I20240426 13:16:32.733353 10585 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20240426 13:16:32.762826 10340 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 8 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 8. Ops: []
I20240426 13:16:32.762984 10491 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 8 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 8. Ops: []
I20240426 13:16:32.763104 10599 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.763248 10599 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.765390 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.765446 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 8 FOLLOWER]: Advancing to term 9
I20240426 13:16:32.765993 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.766085 10585 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.766151 10585 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 8 FOLLOWER]: Advancing to term 9
I20240426 13:16:32.766287 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 9 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:32.766494 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 9 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.766546 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 9 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.766904 10585 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.767074 10340 raft_consensus.cc:2379] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e616a8ecb24e4625bc45ec76a5bd1bfb in current term 9: Already voted for candidate 5c038d4dde124711bb12ef6d54c5b72b in this term.
I20240426 13:16:32.767190 10585 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 9 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.767398 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 9 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.767455 10455 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916
I20240426 13:16:32.767480 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 9 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.767607 10491 raft_consensus.cc:2379] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c038d4dde124711bb12ef6d54c5b72b in current term 9: Already voted for candidate e616a8ecb24e4625bc45ec76a5bd1bfb in this term.
I20240426 13:16:32.767659 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240426 13:16:32.767800 10306 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b; no voters: b67b075c1241471eb4784ac2e1087916, e616a8ecb24e4625bc45ec76a5bd1bfb
I20240426 13:16:32.767926 10585 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20240426 13:16:32.792724 10340 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 9 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 9. Ops: []
I20240426 13:16:32.792726 10491 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 9 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 9. Ops: []
I20240426 13:16:32.793026 10614 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.793144 10599 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.820118 10340 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 9 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 9. Ops: []
I20240426 13:16:32.820351 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.820421 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 9 FOLLOWER]: Advancing to term 10
I20240426 13:16:32.820423 10613 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.820611 10585 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.820672 10585 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 9 FOLLOWER]: Advancing to term 10
I20240426 13:16:32.821041 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.821271 10585 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.821321 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 10 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:32.821599 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 10 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.821619 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 10 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.821763 10340 raft_consensus.cc:2379] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e616a8ecb24e4625bc45ec76a5bd1bfb in current term 10: Already voted for candidate 5c038d4dde124711bb12ef6d54c5b72b in this term.
I20240426 13:16:32.821627 10585 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 10 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.821942 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 10 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.822078 10491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 10 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.822144 10490 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 10 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 10. Ops: []
I20240426 13:16:32.822305 10456 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916
I20240426 13:16:32.822378 10306 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b; no voters: b67b075c1241471eb4784ac2e1087916, e616a8ecb24e4625bc45ec76a5bd1bfb
I20240426 13:16:32.822484 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240426 13:16:32.822526 10585 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20240426 13:16:32.822656 10599 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.842350 10340 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 10 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 10. Ops: [7.3009-7.3009]
I20240426 13:16:32.842350 10490 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 10 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 7. Current term is 10. Ops: [7.3009-7.3009]
I20240426 13:16:32.842720 10613 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.842883 10614 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 7.3008, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.856112 10599 raft_consensus.cc:3032] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 LEADER]: Stepping down as leader of term 7
I20240426 13:16:32.856205 10599 raft_consensus.cc:729] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: b67b075c1241471eb4784ac2e1087916, State: Running, Role: LEADER
I20240426 13:16:32.856308 10599 consensus_queue.cc:260] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3008, Committed index: 3008, Last appended: 7.3009, Last appended by leader: 3009, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.856436 10599 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 7 FOLLOWER]: Advancing to term 8
I20240426 13:16:32.868350 10585 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.868415 10585 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 10 FOLLOWER]: Advancing to term 11
I20240426 13:16:32.868990 10585 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.869292 10585 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 11 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.869561 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 11 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.869596 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 11 candidate_status { last_received { term: 7 index: 3008 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.869657 10415 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 8 FOLLOWER]: Advancing to term 11
I20240426 13:16:32.869690 10490 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 10 FOLLOWER]: Advancing to term 11
I20240426 13:16:32.870342 10415 raft_consensus.cc:2396] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5c038d4dde124711bb12ef6d54c5b72b for term 11 because replica has last-logged OpId of term: 7 index: 3009, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 3008.
I20240426 13:16:32.870414 10490 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c038d4dde124711bb12ef6d54c5b72b in term 11.
I20240426 13:16:32.870659 10306 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b, e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: b67b075c1241471eb4784ac2e1087916
I20240426 13:16:32.870785 10585 raft_consensus.cc:2781] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 11 FOLLOWER]: Leader election won for term 11
I20240426 13:16:32.870867 10585 raft_consensus.cc:686] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 11 LEADER]: Becoming Leader. State: Replica: 5c038d4dde124711bb12ef6d54c5b72b, State: Running, Role: LEADER
I20240426 13:16:32.870976 10585 consensus_queue.cc:237] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3008, Committed index: 3008, Last appended: 7.3008, Last appended by leader: 3008, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.872035 10236 catalog_manager.cc:5484] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b reported cstate change: term changed from 7 to 11, leader changed from b67b075c1241471eb4784ac2e1087916 (127.0.99.130) to 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129). New cstate: current_term: 11 leader_uuid: "5c038d4dde124711bb12ef6d54c5b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:32.885007 10614 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.885071 10614 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 11 FOLLOWER]: Advancing to term 12
I20240426 13:16:32.885711 10614 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.886049 10614 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 12 election: Requested vote from peers 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.886229 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 12 candidate_status { last_received { term: 7 index: 3009 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.886304 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 12 candidate_status { last_received { term: 7 index: 3009 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.886376 10490 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 11 FOLLOWER]: Advancing to term 12
I20240426 13:16:32.886960 10490 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b67b075c1241471eb4784ac2e1087916 in term 12.
I20240426 13:16:32.887214 10382 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b67b075c1241471eb4784ac2e1087916, e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b
I20240426 13:16:32.887362 10614 raft_consensus.cc:2781] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 12 FOLLOWER]: Leader election won for term 12
I20240426 13:16:32.887445 10614 raft_consensus.cc:686] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 12 LEADER]: Becoming Leader. State: Replica: b67b075c1241471eb4784ac2e1087916, State: Running, Role: LEADER
I20240426 13:16:32.887547 10614 consensus_queue.cc:237] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3008, Committed index: 3008, Last appended: 7.3009, Last appended by leader: 3009, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.888530 10236 catalog_manager.cc:5484] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 reported cstate change: term changed from 11 to 12, leader changed from 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129) to b67b075c1241471eb4784ac2e1087916 (127.0.99.130). New cstate: current_term: 12 leader_uuid: "b67b075c1241471eb4784ac2e1087916" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } health_report { overall_health: UNKNOWN } } }
W20240426 13:16:32.892060 10599 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 12 is higher.
I20240426 13:16:32.894920 10490 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 12 FOLLOWER]: Rejecting Update request from peer 5c038d4dde124711bb12ef6d54c5b72b for earlier term 11. Current term is 12. Ops: []
I20240426 13:16:32.895200 10585 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.902753 10415 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 12 LEADER]: Rejecting Update request from peer 5c038d4dde124711bb12ef6d54c5b72b for earlier term 11. Current term is 12. Ops: []
I20240426 13:16:32.903100 10591 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3009, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.903214 10591 consensus_queue.cc:1225] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Peer b67b075c1241471eb4784ac2e1087916 log is divergent from this leader: its last log entry 12.3010 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3008
I20240426 13:16:32.911412 10340 raft_consensus.cc:3032] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 11 LEADER]: Stepping down as leader of term 11
I20240426 13:16:32.911458 10340 raft_consensus.cc:729] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5c038d4dde124711bb12ef6d54c5b72b, State: Running, Role: LEADER
I20240426 13:16:32.911547 10340 consensus_queue.cc:260] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3008, Committed index: 3008, Last appended: 11.3009, Last appended by leader: 3009, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.911651 10340 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 11 FOLLOWER]: Advancing to term 12
I20240426 13:16:32.912199 10340 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 12 FOLLOWER]: Refusing update from remote peer b67b075c1241471eb4784ac2e1087916: Log matching property violated. Preceding OpId in replica: term: 11 index: 3009. Preceding OpId from leader: term: 12 index: 3010. (index mismatch)
I20240426 13:16:32.912492 10614 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3010, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.912571 10614 consensus_queue.cc:1225] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer 5c038d4dde124711bb12ef6d54c5b72b log is divergent from this leader: its last log entry 11.3009 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3008
I20240426 13:16:32.912959 10340 pending_rounds.cc:76] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b: Aborting all ops after (but not including) 3008
I20240426 13:16:32.912995 10340 pending_rounds.cc:97] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b: Aborting uncommitted NO_OP operation due to leader change: 11.3009
I20240426 13:16:32.913012 10340 raft_consensus.cc:2866] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20240426 13:16:32.920904 10490 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 12 FOLLOWER]: Refusing update from remote peer b67b075c1241471eb4784ac2e1087916: Log matching property violated. Preceding OpId in replica: term: 7 index: 3008. Preceding OpId from leader: term: 12 index: 3010. (index mismatch)
I20240426 13:16:32.921243 10614 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3010, Last known committed idx: 3008, Time since last communication: 0.000s
I20240426 13:16:32.945325 10591 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 12 FOLLOWER]: Starting leader election (detected failure of leader b67b075c1241471eb4784ac2e1087916)
I20240426 13:16:32.945384 10591 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 12 FOLLOWER]: Advancing to term 13
I20240426 13:16:32.945902 10591 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.946194 10591 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 13 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.946434 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 13 candidate_status { last_received { term: 12 index: 3010 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.946434 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 13 candidate_status { last_received { term: 12 index: 3010 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.946794 10305 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b; no voters: b67b075c1241471eb4784ac2e1087916, e616a8ecb24e4625bc45ec76a5bd1bfb
I20240426 13:16:32.946916 10591 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20240426 13:16:32.948202 10340 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 13 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 12. Current term is 13. Ops: []
I20240426 13:16:32.948484 10599 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: INVALID_TERM, Last received: 12.3010, Next index: 3011, Last known committed idx: 3010, Time since last communication: 0.000s
I20240426 13:16:32.952919 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 12 FOLLOWER]: Starting leader election (detected failure of leader b67b075c1241471eb4784ac2e1087916)
I20240426 13:16:32.952980 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 12 FOLLOWER]: Advancing to term 13
I20240426 13:16:32.953574 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.953819 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 13 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:32.954098 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 13 candidate_status { last_received { term: 12 index: 3010 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.954131 10340 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 13 candidate_status { last_received { term: 12 index: 3010 } } ignore_live_leader: false dest_uuid: "5c038d4dde124711bb12ef6d54c5b72b"
I20240426 13:16:32.954265 10340 raft_consensus.cc:2379] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e616a8ecb24e4625bc45ec76a5bd1bfb in current term 13: Already voted for candidate 5c038d4dde124711bb12ef6d54c5b72b in this term.
I20240426 13:16:32.954478 10455 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916
I20240426 13:16:32.954608 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20240426 13:16:32.958304 10490 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 13 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 12. Current term is 13. Ops: []
I20240426 13:16:32.958591 10614 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 12.3010, Next index: 3011, Last known committed idx: 3010, Time since last communication: 0.000s
W20240426 13:16:32.961493 10613 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 12 is higher.
I20240426 13:16:32.974398 10340 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 13 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 12. Current term is 13. Ops: []
I20240426 13:16:32.974706 10599 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: INVALID_TERM, Last received: 12.3010, Next index: 3011, Last known committed idx: 3010, Time since last communication: 0.000s
W20240426 13:16:32.983654 10613 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 12 is higher.
I20240426 13:16:32.986765 10490 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 13 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 12. Current term is 13. Ops: []
I20240426 13:16:32.987079 10614 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 12.3010, Next index: 3011, Last known committed idx: 3010, Time since last communication: 0.000s
I20240426 13:16:32.988607 10585 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:32.988663 10585 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 13 FOLLOWER]: Advancing to term 14
I20240426 13:16:32.989286 10585 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.989584 10585 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 14 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:32.989830 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 14 candidate_status { last_received { term: 12 index: 3010 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:32.989831 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "5c038d4dde124711bb12ef6d54c5b72b" candidate_term: 14 candidate_status { last_received { term: 12 index: 3010 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
I20240426 13:16:32.989938 10490 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 13 FOLLOWER]: Advancing to term 14
I20240426 13:16:32.990562 10490 raft_consensus.cc:2454] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c038d4dde124711bb12ef6d54c5b72b in term 14.
I20240426 13:16:32.990778 10306 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5c038d4dde124711bb12ef6d54c5b72b, e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: b67b075c1241471eb4784ac2e1087916
I20240426 13:16:32.990926 10585 raft_consensus.cc:2781] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 14 FOLLOWER]: Leader election won for term 14
I20240426 13:16:32.990999 10585 raft_consensus.cc:686] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 14 LEADER]: Becoming Leader. State: Replica: 5c038d4dde124711bb12ef6d54c5b72b, State: Running, Role: LEADER
I20240426 13:16:32.991107 10585 consensus_queue.cc:237] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3010, Committed index: 3010, Last appended: 12.3010, Last appended by leader: 3010, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:32.992125 10236 catalog_manager.cc:5484] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b reported cstate change: term changed from 12 to 14, leader changed from b67b075c1241471eb4784ac2e1087916 (127.0.99.130) to 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129). New cstate: current_term: 14 leader_uuid: "5c038d4dde124711bb12ef6d54c5b72b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:33.001763 10340 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 14 LEADER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 12. Current term is 14. Ops: []
I20240426 13:16:33.002105 10599 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 }, Status: INVALID_TERM, Last received: 12.3010, Next index: 3011, Last known committed idx: 3010, Time since last communication: 0.000s
I20240426 13:16:33.002203 10599 consensus_queue.cc:1225] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer 5c038d4dde124711bb12ef6d54c5b72b log is divergent from this leader: its last log entry 14.3011 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3010
W20240426 13:16:33.003798 10613 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 12 is higher.
I20240426 13:16:33.011772 10490 raft_consensus.cc:1226] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 14 FOLLOWER]: Rejecting Update request from peer b67b075c1241471eb4784ac2e1087916 for earlier term 12. Current term is 14. Ops: []
I20240426 13:16:33.012044 10599 consensus_queue.cc:1046] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: INVALID_TERM, Last received: 12.3010, Next index: 3011, Last known committed idx: 3010, Time since last communication: 0.000s
I20240426 13:16:33.014353 10490 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 14 FOLLOWER]: Refusing update from remote peer 5c038d4dde124711bb12ef6d54c5b72b: Log matching property violated. Preceding OpId in replica: term: 12 index: 3010. Preceding OpId from leader: term: 14 index: 3011. (index mismatch)
I20240426 13:16:33.014663 10585 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3011, Last known committed idx: 3010, Time since last communication: 0.000s
I20240426 13:16:33.020205 10415 raft_consensus.cc:3032] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 12 LEADER]: Stepping down as leader of term 12
I20240426 13:16:33.020252 10415 raft_consensus.cc:729] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: b67b075c1241471eb4784ac2e1087916, State: Running, Role: LEADER
I20240426 13:16:33.020354 10415 consensus_queue.cc:260] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3010, Committed index: 3010, Last appended: 12.3010, Last appended by leader: 3010, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:33.020462 10415 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 12 FOLLOWER]: Advancing to term 14
I20240426 13:16:33.021014 10415 raft_consensus.cc:1261] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 14 FOLLOWER]: Refusing update from remote peer 5c038d4dde124711bb12ef6d54c5b72b: Log matching property violated. Preceding OpId in replica: term: 12 index: 3010. Preceding OpId from leader: term: 14 index: 3011. (index mismatch)
I20240426 13:16:33.021301 10585 consensus_queue.cc:1035] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3011, Last known committed idx: 3010, Time since last communication: 0.000s
I20240426 13:16:33.022848   398 tablet_server.cc:178] TabletServer@127.0.99.129:0 shutting down...
I20240426 13:16:33.026633   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:33.026777   398 tablet_replica.cc:330] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b: stopping tablet replica
I20240426 13:16:33.026875   398 raft_consensus.cc:2229] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 14 LEADER]: Raft consensus shutting down.
I20240426 13:16:33.044999   398 raft_consensus.cc:2258] T ad1902292fbd419ebbf379c83ee3d9f6 P 5c038d4dde124711bb12ef6d54c5b72b [term 14 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:33.046922 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c038d4dde124711bb12ef6d54c5b72b)
I20240426 13:16:33.047005 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 14 FOLLOWER]: Advancing to term 15
I20240426 13:16:33.047850 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:33.048202 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 15 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
I20240426 13:16:33.048384 10415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" candidate_term: 15 candidate_status { last_received { term: 14 index: 3011 } } ignore_live_leader: false dest_uuid: "b67b075c1241471eb4784ac2e1087916"
W20240426 13:16:33.048539 10455 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
W20240426 13:16:33.049203 10455 leader_election.cc:334] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20240426 13:16:33.049307 10455 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916
I20240426 13:16:33.049429 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20240426 13:16:33.052109 10614 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5c038d4dde124711bb12ef6d54c5b72b)
I20240426 13:16:33.052170 10614 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 14 FOLLOWER]: Advancing to term 15
I20240426 13:16:33.052724 10614 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:33.052996 10614 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 15 election: Requested vote from peers 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747), e616a8ecb24e4625bc45ec76a5bd1bfb (127.0.99.131:43429)
I20240426 13:16:33.053295 10490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad1902292fbd419ebbf379c83ee3d9f6" candidate_uuid: "b67b075c1241471eb4784ac2e1087916" candidate_term: 15 candidate_status { last_received { term: 14 index: 3011 } } ignore_live_leader: false dest_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb"
I20240426 13:16:33.053421 10490 raft_consensus.cc:2379] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate b67b075c1241471eb4784ac2e1087916 in current term 15: Already voted for candidate e616a8ecb24e4625bc45ec76a5bd1bfb in this term.
W20240426 13:16:33.054031 10380 leader_election.cc:334] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747): Network error: Client connection negotiation failed: client connection to 127.0.99.129:37747: connect: Connection refused (error 111)
I20240426 13:16:33.054095 10380 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b67b075c1241471eb4784ac2e1087916; no voters: 5c038d4dde124711bb12ef6d54c5b72b, e616a8ecb24e4625bc45ec76a5bd1bfb
I20240426 13:16:33.054199 10614 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20240426 13:16:33.061775   398 tablet_server.cc:194] TabletServer@127.0.99.129:0 shutdown complete.
I20240426 13:16:33.063374   398 tablet_server.cc:178] TabletServer@127.0.99.130:0 shutting down...
I20240426 13:16:33.066274   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:33.066412   398 tablet_replica.cc:330] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916: stopping tablet replica
I20240426 13:16:33.066516   398 raft_consensus.cc:2229] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 15 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:33.083156   398 raft_consensus.cc:2258] T ad1902292fbd419ebbf379c83ee3d9f6 P b67b075c1241471eb4784ac2e1087916 [term 15 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:33.086104 10605 raft_consensus.cc:482] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:33.086175 10605 raft_consensus.cc:3037] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 15 FOLLOWER]: Advancing to term 16
I20240426 13:16:33.086812 10605 raft_consensus.cc:504] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b67b075c1241471eb4784ac2e1087916" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 38087 } } peers { permanent_uuid: "5c038d4dde124711bb12ef6d54c5b72b" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 37747 } } peers { permanent_uuid: "e616a8ecb24e4625bc45ec76a5bd1bfb" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 43429 } }
I20240426 13:16:33.087091 10605 leader_election.cc:288] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 16 election: Requested vote from peers b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087), 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747)
W20240426 13:16:33.087955 10456 leader_election.cc:334] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer b67b075c1241471eb4784ac2e1087916 (127.0.99.130:38087): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20240426 13:16:33.088299 10455 leader_election.cc:334] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 5c038d4dde124711bb12ef6d54c5b72b (127.0.99.129:37747): Network error: Client connection negotiation failed: client connection to 127.0.99.129:37747: connect: Connection refused (error 111)
I20240426 13:16:33.088383 10455 leader_election.cc:302] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e616a8ecb24e4625bc45ec76a5bd1bfb; no voters: 5c038d4dde124711bb12ef6d54c5b72b, b67b075c1241471eb4784ac2e1087916
I20240426 13:16:33.088524 10605 raft_consensus.cc:2726] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20240426 13:16:33.099695   398 tablet_server.cc:194] TabletServer@127.0.99.130:0 shutdown complete.
I20240426 13:16:33.101162   398 tablet_server.cc:178] TabletServer@127.0.99.131:0 shutting down...
I20240426 13:16:33.104266   398 ts_tablet_manager.cc:1470] Shutting down tablet manager...
I20240426 13:16:33.104415   398 tablet_replica.cc:330] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb: stopping tablet replica
I20240426 13:16:33.104518   398 raft_consensus.cc:2229] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 16 FOLLOWER]: Raft consensus shutting down.
I20240426 13:16:33.104599   398 raft_consensus.cc:2258] T ad1902292fbd419ebbf379c83ee3d9f6 P e616a8ecb24e4625bc45ec76a5bd1bfb [term 16 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:33.114249   398 tablet_server.cc:194] TabletServer@127.0.99.131:0 shutdown complete.
I20240426 13:16:33.115639   398 master.cc:537] Master@127.0.99.190:42111 shutting down...
I20240426 13:16:33.119366   398 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 1 LEADER]: Raft consensus shutting down.
I20240426 13:16:33.119483   398 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20240426 13:16:33.119556   398 tablet_replica.cc:330] T 00000000000000000000000000000000 P cfe79240f4974fd399668ae5ea8234d5: stopping tablet replica
I20240426 13:16:33.131394   398 master.cc:552] Master@127.0.99.190:42111 shutdown complete.
[       OK ] TxnParticipantElectionStormITest.TestTxnSystemClientRetriesThroughStorm (10978 ms)
[----------] 2 tests from TxnParticipantElectionStormITest (27418 ms total)

[----------] 2 tests from ShouldGCWals/ParticipantCopyITest
[ RUN      ] ShouldGCWals/ParticipantCopyITest.TestCopyParticipantOps/0
I20240426 13:16:33.139026   398 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.99.190:33695
I20240426 13:16:33.139310   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:33.140957 10619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:33.141958 10620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:33.142871 10622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:33.143604   398 server_base.cc:1034] running on GCE node
I20240426 13:16:33.143723   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:33.143754   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:33.143772   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137393143772 us; error 0 us; skew 500 ppm
I20240426 13:16:33.144487   398 webserver.cc:460] Webserver started at http://127.0.99.190:44837/ using document root <none> and password file <none>
I20240426 13:16:33.144604   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:33.144644   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:33.144706   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:33.145007   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/master-0-root/instance:
uuid: "07187a9d43204769948faa5559db4527"
format_stamp: "Formatted at 2024-04-26 13:16:33 on dist-test-slave-2jsl"
I20240426 13:16:33.146319   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20240426 13:16:33.147119 10627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.147316   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:33.147374   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/master-0-root
uuid: "07187a9d43204769948faa5559db4527"
format_stamp: "Formatted at 2024-04-26 13:16:33 on dist-test-slave-2jsl"
I20240426 13:16:33.147433   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:33.156738   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:33.156981   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:33.160409   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.190:33695
I20240426 13:16:33.161888 10679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.190:33695 every 8 connection(s)
I20240426 13:16:33.162187 10680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:33.163501 10680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527: Bootstrap starting.
I20240426 13:16:33.164083 10680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:33.164806 10680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527: No bootstrap required, opened a new log
I20240426 13:16:33.165072 10680 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07187a9d43204769948faa5559db4527" member_type: VOTER }
I20240426 13:16:33.165135 10680 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:33.165154 10680 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07187a9d43204769948faa5559db4527, State: Initialized, Role: FOLLOWER
I20240426 13:16:33.165228 10680 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07187a9d43204769948faa5559db4527" member_type: VOTER }
I20240426 13:16:33.165272 10680 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20240426 13:16:33.165292 10680 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20240426 13:16:33.165323 10680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:33.165817 10680 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07187a9d43204769948faa5559db4527" member_type: VOTER }
I20240426 13:16:33.165903 10680 leader_election.cc:302] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07187a9d43204769948faa5559db4527; no voters: 
I20240426 13:16:33.166049 10680 leader_election.cc:288] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20240426 13:16:33.166133 10683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:16:33.166328 10683 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [term 1 LEADER]: Becoming Leader. State: Replica: 07187a9d43204769948faa5559db4527, State: Running, Role: LEADER
I20240426 13:16:33.166427 10680 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [sys.catalog]: configured and running, proceeding with master startup.
I20240426 13:16:33.166467 10683 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07187a9d43204769948faa5559db4527" member_type: VOTER }
I20240426 13:16:33.166855 10684 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07187a9d43204769948faa5559db4527" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07187a9d43204769948faa5559db4527" member_type: VOTER } }
I20240426 13:16:33.166893 10685 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07187a9d43204769948faa5559db4527. Latest consensus state: current_term: 1 leader_uuid: "07187a9d43204769948faa5559db4527" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07187a9d43204769948faa5559db4527" member_type: VOTER } }
I20240426 13:16:33.167033 10685 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [sys.catalog]: This master's current role is: LEADER
I20240426 13:16:33.167021 10684 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527 [sys.catalog]: This master's current role is: LEADER
I20240426 13:16:33.167639 10690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20240426 13:16:33.168491 10690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20240426 13:16:33.168653   398 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20240426 13:16:33.170053 10690 catalog_manager.cc:1347] Generated new cluster ID: 959397af22b84a288d8258f452923cac
I20240426 13:16:33.170105 10690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20240426 13:16:33.184345 10690 catalog_manager.cc:1370] Generated new certificate authority record
I20240426 13:16:33.184777 10690 catalog_manager.cc:1504] Loading token signing keys...
I20240426 13:16:33.188656 10690 catalog_manager.cc:5857] T 00000000000000000000000000000000 P 07187a9d43204769948faa5559db4527: Generated new TSK 0
I20240426 13:16:33.188791 10690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20240426 13:16:33.200565   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:33.202018 10701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:33.202878 10704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:33.203698 10702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:33.204478   398 server_base.cc:1034] running on GCE node
I20240426 13:16:33.204599   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:33.204633   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:33.204646   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137393204646 us; error 0 us; skew 500 ppm
I20240426 13:16:33.205291   398 webserver.cc:460] Webserver started at http://127.0.99.129:33163/ using document root <none> and password file <none>
I20240426 13:16:33.205411   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:33.205449   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:33.205497   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:33.205780   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-0-root/instance:
uuid: "2c806d4334ed4b06a3222a351ecf0833"
format_stamp: "Formatted at 2024-04-26 13:16:33 on dist-test-slave-2jsl"
I20240426 13:16:33.207005   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:16:33.207700 10709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.207898   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:16:33.207957   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-0-root
uuid: "2c806d4334ed4b06a3222a351ecf0833"
format_stamp: "Formatted at 2024-04-26 13:16:33 on dist-test-slave-2jsl"
I20240426 13:16:33.208014   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:33.213255   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:33.213493   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:33.213755   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:33.214108   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:33.214141   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.214179   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:33.214205   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.217572   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.129:46767
I20240426 13:16:33.217593 10772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.129:46767 every 8 connection(s)
I20240426 13:16:33.218226   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:33.220374 10778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:33.220595 10777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:33.221282 10780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:33.222085   398 server_base.cc:1034] running on GCE node
I20240426 13:16:33.222216   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:33.222249   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:33.222260   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137393222260 us; error 0 us; skew 500 ppm
I20240426 13:16:33.222575 10773 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33695
I20240426 13:16:33.222651 10773 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:33.222807 10773 heartbeater.cc:511] Master 127.0.99.190:33695 requested a full tablet report, sending...
I20240426 13:16:33.223235   398 webserver.cc:460] Webserver started at http://127.0.99.130:39653/ using document root <none> and password file <none>
I20240426 13:16:33.223356   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:33.223388 10644 ts_manager.cc:194] Registered new tserver with Master: 2c806d4334ed4b06a3222a351ecf0833 (127.0.99.129:46767)
I20240426 13:16:33.223395   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:33.223541   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:33.223807   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-1-root/instance:
uuid: "1d95e865423d4797aa98bed1e6fa621d"
format_stamp: "Formatted at 2024-04-26 13:16:33 on dist-test-slave-2jsl"
I20240426 13:16:33.224120 10644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37240
I20240426 13:16:33.225101   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:33.226258 10785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.226645   398 fs_manager.cc:725] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:33.226718   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-1-root
uuid: "1d95e865423d4797aa98bed1e6fa621d"
format_stamp: "Formatted at 2024-04-26 13:16:33 on dist-test-slave-2jsl"
I20240426 13:16:33.226797   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:33.240048   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:33.240274   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:33.240475   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:33.240800   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:33.240831   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.240859   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:33.240885   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.244627   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.130:43367
I20240426 13:16:33.244643 10848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.130:43367 every 8 connection(s)
I20240426 13:16:33.245317   398 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20240426 13:16:33.247041 10852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:33.248116 10853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20240426 13:16:33.248603 10855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20240426 13:16:33.248858 10849 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33695
I20240426 13:16:33.248922 10849 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:33.249078 10849 heartbeater.cc:511] Master 127.0.99.190:33695 requested a full tablet report, sending...
I20240426 13:16:33.249470   398 server_base.cc:1034] running on GCE node
I20240426 13:16:33.249637 10644 ts_manager.cc:194] Registered new tserver with Master: 1d95e865423d4797aa98bed1e6fa621d (127.0.99.130:43367)
I20240426 13:16:33.249639   398 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20240426 13:16:33.249737   398 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20240426 13:16:33.249768   398 hybrid_clock.cc:648] HybridClock initialized: now 1714137393249766 us; error 0 us; skew 500 ppm
I20240426 13:16:33.250264 10644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37244
I20240426 13:16:33.250591   398 webserver.cc:460] Webserver started at http://127.0.99.131:41709/ using document root <none> and password file <none>
I20240426 13:16:33.250721   398 fs_manager.cc:359] Metadata directory not provided
I20240426 13:16:33.250770   398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20240426 13:16:33.250838   398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20240426 13:16:33.251133   398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-2-root/instance:
uuid: "1e6fd25eb2a64d94a05b6e057222181c"
format_stamp: "Formatted at 2024-04-26 13:16:33 on dist-test-slave-2jsl"
I20240426 13:16:33.252239   398 fs_manager.cc:691] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20240426 13:16:33.252908 10860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.253082   398 fs_manager.cc:725] Time spent opening block manager: real 0.000s	user 0.001s	sys 0.000s
I20240426 13:16:33.253131   398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-2-root
uuid: "1e6fd25eb2a64d94a05b6e057222181c"
format_stamp: "Formatted at 2024-04-26 13:16:33 on dist-test-slave-2jsl"
I20240426 13:16:33.253188   398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20240426 13:16:33.259194   398 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20240426 13:16:33.259570   398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20240426 13:16:33.260082   398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20240426 13:16:33.260841   398 ts_tablet_manager.cc:559] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20240426 13:16:33.260913   398 ts_tablet_manager.cc:505] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.260962   398 ts_tablet_manager.cc:590] Registered 0 tablets
I20240426 13:16:33.260995   398 ts_tablet_manager.cc:569] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20240426 13:16:33.264976   398 rpc_server.cc:307] RPC server started. Bound to: 127.0.99.131:46833
I20240426 13:16:33.265012 10923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.99.131:46833 every 8 connection(s)
I20240426 13:16:33.268611 10924 heartbeater.cc:347] Connected to a master server at 127.0.99.190:33695
I20240426 13:16:33.268679 10924 heartbeater.cc:464] Registering TS with master...
I20240426 13:16:33.268872 10924 heartbeater.cc:511] Master 127.0.99.190:33695 requested a full tablet report, sending...
I20240426 13:16:33.269387 10644 ts_manager.cc:194] Registered new tserver with Master: 1e6fd25eb2a64d94a05b6e057222181c (127.0.99.131:46833)
I20240426 13:16:33.269851   398 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.004570302s
I20240426 13:16:33.269960 10644 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37258
I20240426 13:16:33.270460   398 test_util.cc:268] Using random seed: -1234432325
I20240426 13:16:33.276181 10644 catalog_manager.cc:1904] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37266:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20240426 13:16:33.276470 10644 catalog_manager.cc:6843] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20240426 13:16:33.283535 10813 tablet_service.cc:1467] Processing CreateTablet for tablet e5d2e147c8c94844b14f3a93e5e47e6c (DEFAULT_TABLE table=test-workload [id=af31b8acdcd048bc9a9e0f932a14ef7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:33.283769 10813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d2e147c8c94844b14f3a93e5e47e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:33.284757 10888 tablet_service.cc:1467] Processing CreateTablet for tablet e5d2e147c8c94844b14f3a93e5e47e6c (DEFAULT_TABLE table=test-workload [id=af31b8acdcd048bc9a9e0f932a14ef7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:33.284945 10888 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d2e147c8c94844b14f3a93e5e47e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:33.285492 10939 tablet_bootstrap.cc:492] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d: Bootstrap starting.
I20240426 13:16:33.285635 10737 tablet_service.cc:1467] Processing CreateTablet for tablet e5d2e147c8c94844b14f3a93e5e47e6c (DEFAULT_TABLE table=test-workload [id=af31b8acdcd048bc9a9e0f932a14ef7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20240426 13:16:33.285827 10737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5d2e147c8c94844b14f3a93e5e47e6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20240426 13:16:33.286345 10939 tablet_bootstrap.cc:654] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:33.286686 10940 tablet_bootstrap.cc:492] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c: Bootstrap starting.
I20240426 13:16:33.287256 10939 tablet_bootstrap.cc:492] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d: No bootstrap required, opened a new log
I20240426 13:16:33.287322 10939 ts_tablet_manager.cc:1367] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d: Time spent bootstrapping tablet: real 0.002s	user 0.000s	sys 0.001s
I20240426 13:16:33.287360 10940 tablet_bootstrap.cc:654] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:33.287719 10939 raft_consensus.cc:348] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
I20240426 13:16:33.287808 10939 raft_consensus.cc:374] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:33.287824 10942 tablet_bootstrap.cc:492] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833: Bootstrap starting.
I20240426 13:16:33.287846 10939 raft_consensus.cc:729] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d95e865423d4797aa98bed1e6fa621d, State: Initialized, Role: FOLLOWER
I20240426 13:16:33.287990 10939 consensus_queue.cc:260] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
I20240426 13:16:33.288286 10939 ts_tablet_manager.cc:1398] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d: Time spent starting tablet: real 0.001s	user 0.000s	sys 0.001s
I20240426 13:16:33.288290 10849 heartbeater.cc:503] Master 127.0.99.190:33695 was elected leader, sending a full tablet report...
I20240426 13:16:33.288367 10940 tablet_bootstrap.cc:492] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c: No bootstrap required, opened a new log
I20240426 13:16:33.288432 10940 ts_tablet_manager.cc:1367] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c: Time spent bootstrapping tablet: real 0.002s	user 0.001s	sys 0.000s
I20240426 13:16:33.289124 10940 raft_consensus.cc:348] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
I20240426 13:16:33.289243 10942 tablet_bootstrap.cc:654] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833: Neither blocks nor log segments found. Creating new log.
I20240426 13:16:33.289263 10940 raft_consensus.cc:374] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:33.289296 10940 raft_consensus.cc:729] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6fd25eb2a64d94a05b6e057222181c, State: Initialized, Role: FOLLOWER
I20240426 13:16:33.289451 10940 consensus_queue.cc:260] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
I20240426 13:16:33.289804 10940 ts_tablet_manager.cc:1398] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:33.289861 10924 heartbeater.cc:503] Master 127.0.99.190:33695 was elected leader, sending a full tablet report...
I20240426 13:16:33.290449 10942 tablet_bootstrap.cc:492] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833: No bootstrap required, opened a new log
I20240426 13:16:33.290534 10942 ts_tablet_manager.cc:1367] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833: Time spent bootstrapping tablet: real 0.003s	user 0.001s	sys 0.000s
I20240426 13:16:33.290931 10942 raft_consensus.cc:348] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
I20240426 13:16:33.291054 10942 raft_consensus.cc:374] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20240426 13:16:33.291100 10942 raft_consensus.cc:729] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c806d4334ed4b06a3222a351ecf0833, State: Initialized, Role: FOLLOWER
I20240426 13:16:33.291200 10942 consensus_queue.cc:260] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
I20240426 13:16:33.291471 10942 ts_tablet_manager.cc:1398] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833: Time spent starting tablet: real 0.001s	user 0.001s	sys 0.000s
I20240426 13:16:33.291805 10773 heartbeater.cc:503] Master 127.0.99.190:33695 was elected leader, sending a full tablet report...
I20240426 13:16:33.296480 10947 raft_consensus.cc:482] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20240426 13:16:33.296547 10947 raft_consensus.cc:504] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
W20240426 13:16:33.296805 10925 tablet.cc:2369] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20240426 13:16:33.297195 10947 leader_election.cc:288] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d95e865423d4797aa98bed1e6fa621d (127.0.99.130:43367), 1e6fd25eb2a64d94a05b6e057222181c (127.0.99.131:46833)
I20240426 13:16:33.299724 10823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d2e147c8c94844b14f3a93e5e47e6c" candidate_uuid: "2c806d4334ed4b06a3222a351ecf0833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d95e865423d4797aa98bed1e6fa621d" is_pre_election: true
I20240426 13:16:33.299845 10823 raft_consensus.cc:2454] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c806d4334ed4b06a3222a351ecf0833 in term 0.
I20240426 13:16:33.299921 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d2e147c8c94844b14f3a93e5e47e6c" candidate_uuid: "2c806d4334ed4b06a3222a351ecf0833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6fd25eb2a64d94a05b6e057222181c" is_pre_election: true
I20240426 13:16:33.300048 10898 raft_consensus.cc:2454] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c806d4334ed4b06a3222a351ecf0833 in term 0.
I20240426 13:16:33.300148 10711 leader_election.cc:302] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d95e865423d4797aa98bed1e6fa621d, 2c806d4334ed4b06a3222a351ecf0833; no voters: 
I20240426 13:16:33.300371 10947 raft_consensus.cc:2781] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20240426 13:16:33.300419 10947 raft_consensus.cc:482] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20240426 13:16:33.300443 10947 raft_consensus.cc:3037] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:33.300995 10947 raft_consensus.cc:504] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
I20240426 13:16:33.301242 10947 leader_election.cc:288] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [CANDIDATE]: Term 1 election: Requested vote from peers 1d95e865423d4797aa98bed1e6fa621d (127.0.99.130:43367), 1e6fd25eb2a64d94a05b6e057222181c (127.0.99.131:46833)
I20240426 13:16:33.301476 10898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d2e147c8c94844b14f3a93e5e47e6c" candidate_uuid: "2c806d4334ed4b06a3222a351ecf0833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e6fd25eb2a64d94a05b6e057222181c"
I20240426 13:16:33.301555 10898 raft_consensus.cc:3037] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:33.302073 10823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5d2e147c8c94844b14f3a93e5e47e6c" candidate_uuid: "2c806d4334ed4b06a3222a351ecf0833" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d95e865423d4797aa98bed1e6fa621d"
I20240426 13:16:33.302165 10823 raft_consensus.cc:3037] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d [term 0 FOLLOWER]: Advancing to term 1
I20240426 13:16:33.302309 10898 raft_consensus.cc:2454] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c806d4334ed4b06a3222a351ecf0833 in term 1.
I20240426 13:16:33.302554 10713 leader_election.cc:302] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e6fd25eb2a64d94a05b6e057222181c, 2c806d4334ed4b06a3222a351ecf0833; no voters: 
I20240426 13:16:33.302810 10947 raft_consensus.cc:2781] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 1 FOLLOWER]: Leader election won for term 1
I20240426 13:16:33.303037 10823 raft_consensus.cc:2454] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c806d4334ed4b06a3222a351ecf0833 in term 1.
I20240426 13:16:33.302978 10947 raft_consensus.cc:686] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [term 1 LEADER]: Becoming Leader. State: Replica: 2c806d4334ed4b06a3222a351ecf0833, State: Running, Role: LEADER
I20240426 13:16:33.303184 10947 consensus_queue.cc:237] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } }
I20240426 13:16:33.304291 10644 catalog_manager.cc:5484] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c806d4334ed4b06a3222a351ecf0833 (127.0.99.129). New cstate: current_term: 1 leader_uuid: "2c806d4334ed4b06a3222a351ecf0833" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c806d4334ed4b06a3222a351ecf0833" member_type: VOTER last_known_addr { host: "127.0.99.129" port: 46767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 } health_report { overall_health: UNKNOWN } } }
I20240426 13:16:33.329905 10823 raft_consensus.cc:1261] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d [term 1 FOLLOWER]: Refusing update from remote peer 2c806d4334ed4b06a3222a351ecf0833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:16:33.330039 10898 raft_consensus.cc:1261] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c [term 1 FOLLOWER]: Refusing update from remote peer 2c806d4334ed4b06a3222a351ecf0833: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20240426 13:16:33.330320 10947 consensus_queue.cc:1035] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d95e865423d4797aa98bed1e6fa621d" member_type: VOTER last_known_addr { host: "127.0.99.130" port: 43367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:16:33.330693 10947 consensus_queue.cc:1035] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e6fd25eb2a64d94a05b6e057222181c" member_type: VOTER last_known_addr { host: "127.0.99.131" port: 46833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20240426 13:16:33.933507 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:33.934582 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:33.950027 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:34.172909 10633 maintenance_manager.cc:419] P 07187a9d43204769948faa5559db4527: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20240426 13:16:34.230772 10632 maintenance_manager.cc:643] P 07187a9d43204769948faa5559db4527: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.058s	user 0.012s	sys 0.001s Metrics: {"bytes_written":6631,"cfile_init":1,"drs_written":1,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4,"lbm_write_time_us":579,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"thread_start_us":96,"threads_started":1}
I20240426 13:16:34.231423 10633 maintenance_manager.cc:419] P 07187a9d43204769948faa5559db4527: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 485 bytes on disk
I20240426 13:16:34.231954 10632 maintenance_manager.cc:643] P 07187a9d43204769948faa5559db4527: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":55,"lbm_reads_lt_1ms":4}
I20240426 13:16:34.343180 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.409s	user 0.193s	sys 0.000s Metrics: {"bytes_written":273758,"cfile_init":1,"compiler_manager_pool.queue_time_us":7978,"compiler_manager_pool.run_cpu_time_us":217574,"compiler_manager_pool.run_wall_time_us":270608,"drs_written":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1078,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9500,"spinlock_wait_cycles":3328,"thread_start_us":139,"threads_started":2}
I20240426 13:16:34.343729 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 88764 bytes on disk
I20240426 13:16:34.344079 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4}
I20240426 13:16:34.495216 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.560s	user 0.199s	sys 0.000s Metrics: {"bytes_written":269845,"cfile_init":1,"compiler_manager_pool.queue_time_us":278157,"compiler_manager_pool.run_cpu_time_us":18,"compiler_manager_pool.run_wall_time_us":18,"dirs.queue_time_us":15914,"dirs.run_cpu_time_us":128,"dirs.run_wall_time_us":1853,"drs_written":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1125,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9350,"thread_start_us":86,"threads_started":1}
I20240426 13:16:34.495689 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 87299 bytes on disk
I20240426 13:16:34.496731 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4}
I20240426 13:16:34.527076 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.577s	user 0.191s	sys 0.004s Metrics: {"bytes_written":273758,"cfile_init":1,"compiler_manager_pool.queue_time_us":262749,"compiler_manager_pool.run_cpu_time_us":15,"compiler_manager_pool.run_wall_time_us":15,"dirs.queue_time_us":5522,"dirs.run_cpu_time_us":119,"dirs.run_wall_time_us":3295,"drs_written":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1116,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9500,"thread_start_us":72,"threads_started":1}
I20240426 13:16:34.527576 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 88764 bytes on disk
I20240426 13:16:34.527861 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4}
I20240426 13:16:34.732192 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:34.734165 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:34.738574 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:34.965245 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.233s	user 0.192s	sys 0.000s Metrics: {"bytes_written":272538,"cfile_init":1,"dirs.queue_time_us":34,"dirs.run_cpu_time_us":141,"dirs.run_wall_time_us":821,"drs_written":1,"lbm_read_time_us":133,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1277,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9450}
I20240426 13:16:34.966135 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 88097 bytes on disk
I20240426 13:16:34.966562 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4}
I20240426 13:16:34.966997 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014773
I20240426 13:16:34.993741 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.259s	user 0.180s	sys 0.000s Metrics: {"bytes_written":271149,"cfile_init":1,"dirs.queue_time_us":34,"dirs.run_cpu_time_us":105,"dirs.run_wall_time_us":616,"drs_written":1,"lbm_read_time_us":51,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1141,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9400}
I20240426 13:16:34.994249 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014773
I20240426 13:16:35.201563 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.463s	user 0.170s	sys 0.000s Metrics: {"bytes_written":271149,"cfile_init":1,"dirs.queue_time_us":16793,"dirs.run_cpu_time_us":143,"dirs.run_wall_time_us":4527,"drs_written":1,"lbm_read_time_us":46,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1139,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9400,"thread_start_us":81,"threads_started":1}
I20240426 13:16:35.202322 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014773
I20240426 13:16:35.248261 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.281s	user 0.274s	sys 0.000s Metrics: {"bytes_written":534416,"cfile_cache_hit":12,"cfile_cache_hit_bytes":412757,"cfile_cache_miss":40,"cfile_cache_miss_bytes":922381,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":30,"dirs.run_cpu_time_us":87,"dirs.run_wall_time_us":531,"drs_written":1,"lbm_read_time_us":1015,"lbm_reads_lt_1ms":60,"lbm_write_time_us":1681,"lbm_writes_lt_1ms":42,"num_input_rowsets":2,"peak_mem_usage":281579,"rows_written":18800}
I20240426 13:16:35.248770 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 180525 bytes on disk
I20240426 13:16:35.249166 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":46,"lbm_reads_lt_1ms":4}
I20240426 13:16:35.280362 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.286s	user 0.271s	sys 0.002s Metrics: {"bytes_written":541058,"cfile_cache_hit":12,"cfile_cache_hit_bytes":415497,"cfile_cache_miss":40,"cfile_cache_miss_bytes":926757,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":72,"dirs.run_cpu_time_us":88,"dirs.run_wall_time_us":554,"drs_written":1,"lbm_read_time_us":1065,"lbm_reads_lt_1ms":64,"lbm_write_time_us":1623,"lbm_writes_lt_1ms":43,"num_input_rowsets":2,"peak_mem_usage":283109,"rows_written":18900,"thread_start_us":339,"threads_started":5}
I20240426 13:16:35.280721 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 181594 bytes on disk
I20240426 13:16:35.281075 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":35,"lbm_reads_lt_1ms":4}
I20240426 13:16:35.512181 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.310s	user 0.281s	sys 0.004s Metrics: {"bytes_written":541058,"cfile_cache_hit":12,"cfile_cache_hit_bytes":415497,"cfile_cache_miss":40,"cfile_cache_miss_bytes":926757,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":202,"dirs.run_cpu_time_us":102,"dirs.run_wall_time_us":681,"drs_written":1,"lbm_read_time_us":1151,"lbm_reads_lt_1ms":64,"lbm_write_time_us":2203,"lbm_writes_lt_1ms":43,"num_input_rowsets":2,"peak_mem_usage":283109,"rows_written":18900,"thread_start_us":292,"threads_started":5}
I20240426 13:16:35.512732 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 181594 bytes on disk
I20240426 13:16:35.513099 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4}
I20240426 13:16:35.730356 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:35.734479 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:35.744554 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:35.968325 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.223s	user 0.166s	sys 0.004s Metrics: {"bytes_written":272564,"cfile_init":1,"dirs.queue_time_us":917,"dirs.run_cpu_time_us":137,"dirs.run_wall_time_us":1412,"drs_written":1,"lbm_read_time_us":45,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1096,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9450,"thread_start_us":54,"threads_started":1}
I20240426 13:16:35.969174 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 87472 bytes on disk
I20240426 13:16:35.969604 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4}
I20240426 13:16:35.975003 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014678
I20240426 13:16:36.156284 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.426s	user 0.179s	sys 0.000s Metrics: {"bytes_written":271128,"cfile_init":1,"dirs.queue_time_us":12552,"dirs.run_cpu_time_us":140,"dirs.run_wall_time_us":985,"drs_written":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1208,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9400,"thread_start_us":67,"threads_started":1}
I20240426 13:16:36.156831 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014676
I20240426 13:16:36.179579 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.445s	user 0.174s	sys 0.001s Metrics: {"bytes_written":271128,"cfile_init":1,"dirs.queue_time_us":13293,"dirs.run_cpu_time_us":100,"dirs.run_wall_time_us":7974,"drs_written":1,"lbm_read_time_us":40,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1125,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9400,"thread_start_us":64,"threads_started":1}
I20240426 13:16:36.180133 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014676
I20240426 13:16:36.624495 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.467s	user 0.411s	sys 0.004s Metrics: {"bytes_written":805744,"cfile_cache_hit":10,"cfile_cache_hit_bytes":439512,"cfile_cache_miss":52,"cfile_cache_miss_bytes":1572168,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":73,"dirs.run_cpu_time_us":98,"dirs.run_wall_time_us":799,"drs_written":1,"lbm_read_time_us":1691,"lbm_reads_lt_1ms":76,"lbm_write_time_us":2382,"lbm_writes_lt_1ms":54,"mutex_wait_us":3,"num_input_rowsets":2,"peak_mem_usage":426441,"rows_written":28300,"thread_start_us":290,"threads_started":5}
I20240426 13:16:36.625144 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 279208 bytes on disk
I20240426 13:16:36.625478 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4}
I20240426 13:16:36.676172 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.701s	user 0.399s	sys 0.008s Metrics: {"bytes_written":804519,"cfile_cache_hit":13,"cfile_cache_hit_bytes":583193,"cfile_cache_miss":49,"cfile_cache_miss_bytes":1425139,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":12865,"dirs.run_cpu_time_us":133,"dirs.run_wall_time_us":8585,"drs_written":1,"lbm_read_time_us":1489,"lbm_reads_lt_1ms":69,"lbm_write_time_us":6190,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":52,"num_input_rowsets":2,"peak_mem_usage":425676,"rows_written":28250,"spinlock_wait_cycles":1280,"thread_start_us":66,"threads_started":1}
I20240426 13:16:36.676734 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 278625 bytes on disk
I20240426 13:16:36.678714 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4}
I20240426 13:16:36.794668 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:36.807615 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.627s	user 0.405s	sys 0.012s Metrics: {"bytes_written":805744,"cfile_cache_hit":12,"cfile_cache_hit_bytes":552990,"cfile_cache_miss":50,"cfile_cache_miss_bytes":1458690,"cfile_init":6,"delta_iterators_relevant":4,"drs_written":1,"lbm_read_time_us":3423,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":73,"lbm_write_time_us":5200,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":53,"num_input_rowsets":2,"peak_mem_usage":426441,"rows_written":28300,"spinlock_wait_cycles":384,"thread_start_us":508,"threads_started":7}
I20240426 13:16:36.808130 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 279208 bytes on disk
I20240426 13:16:36.808557 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":38,"lbm_reads_lt_1ms":4}
I20240426 13:16:36.824376 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:36.849762 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:37.159567 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.365s	user 0.176s	sys 0.000s Metrics: {"bytes_written":271196,"cfile_init":1,"dirs.queue_time_us":150,"dirs.run_cpu_time_us":122,"dirs.run_wall_time_us":1052,"drs_written":1,"lbm_read_time_us":44,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1075,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9400}
I20240426 13:16:37.160101 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014276
I20240426 13:16:37.258289 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.408s	user 0.183s	sys 0.001s Metrics: {"bytes_written":272479,"cfile_init":1,"dirs.queue_time_us":32,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":798,"drs_written":1,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1160,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9450}
I20240426 13:16:37.259516 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014276
I20240426 13:16:37.378717 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.554s	user 0.178s	sys 0.000s Metrics: {"bytes_written":272569,"cfile_init":1,"dirs.queue_time_us":14589,"dirs.run_cpu_time_us":140,"dirs.run_wall_time_us":875,"drs_written":1,"lbm_read_time_us":42,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4822,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"rows_written":9450,"spinlock_wait_cycles":193408,"thread_start_us":59,"threads_started":1}
I20240426 13:16:37.379334 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): free 1069056 bytes of WAL
I20240426 13:16:37.379608 10790 log_reader.cc:386] T e5d2e147c8c94844b14f3a93e5e47e6c: removed 1 log segments from log reader
I20240426 13:16:37.379724 10790 log.cc:1076] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d: Deleting log segment in path: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-1-root/wals/e5d2e147c8c94844b14f3a93e5e47e6c/wal-000000001 (ops 1-614)
I20240426 13:16:37.380241 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20240426 13:16:37.380544 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 87581 bytes on disk
I20240426 13:16:37.380890 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20240426 13:16:37.381273 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014277
I20240426 13:16:37.901243 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.642s	user 0.529s	sys 0.011s Metrics: {"bytes_written":1072940,"cfile_cache_hit":12,"cfile_cache_hit_bytes":606062,"cfile_cache_miss":62,"cfile_cache_miss_bytes":2092940,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":18775,"dirs.run_cpu_time_us":141,"dirs.run_wall_time_us":1042,"drs_written":1,"lbm_read_time_us":4496,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":85,"lbm_write_time_us":3198,"lbm_writes_lt_1ms":64,"num_input_rowsets":2,"peak_mem_usage":570526,"rows_written":37750,"thread_start_us":77,"threads_started":1}
I20240426 13:16:37.901829 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): free 1051501 bytes of WAL
I20240426 13:16:37.902081 10865 log_reader.cc:386] T e5d2e147c8c94844b14f3a93e5e47e6c: removed 1 log segments from log reader
I20240426 13:16:37.902161 10865 log.cc:1076] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c: Deleting log segment in path: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-2-root/wals/e5d2e147c8c94844b14f3a93e5e47e6c/wal-000000001 (ops 1-595)
I20240426 13:16:37.902590 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20240426 13:16:37.902815 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 379879 bytes on disk
I20240426 13:16:37.903118 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":41,"lbm_reads_lt_1ms":4}
I20240426 13:16:37.922099 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.541s	user 0.528s	sys 0.004s Metrics: {"bytes_written":1071401,"cfile_cache_hit":12,"cfile_cache_hit_bytes":605822,"cfile_cache_miss":62,"cfile_cache_miss_bytes":2089720,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":83,"dirs.run_cpu_time_us":83,"dirs.run_wall_time_us":1830,"drs_written":1,"lbm_read_time_us":1959,"lbm_reads_lt_1ms":82,"lbm_write_time_us":3106,"lbm_writes_lt_1ms":64,"num_input_rowsets":2,"peak_mem_usage":569766,"rows_written":37700,"thread_start_us":337,"threads_started":5}
I20240426 13:16:37.922555 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 379406 bytes on disk
I20240426 13:16:37.922927 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20240426 13:16:38.017064 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:38.029440 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:38.244030 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.227s	user 0.158s	sys 0.012s Metrics: {"bytes_written":271042,"cfile_init":1,"dirs.queue_time_us":31,"dirs.run_cpu_time_us":136,"dirs.run_wall_time_us":951,"drs_written":1,"lbm_read_time_us":48,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1269,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9400}
I20240426 13:16:38.244851 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 87497 bytes on disk
I20240426 13:16:38.245270 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20240426 13:16:38.245683 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014451
I20240426 13:16:38.348553 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 1.176s	user 0.565s	sys 0.018s Metrics: {"bytes_written":1071401,"cfile_cache_hit":11,"cfile_cache_hit_bytes":522369,"cfile_cache_miss":63,"cfile_cache_miss_bytes":2173131,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":963,"dirs.run_cpu_time_us":136,"dirs.run_wall_time_us":2593,"drs_written":1,"lbm_read_time_us":2120,"lbm_reads_lt_1ms":87,"lbm_write_time_us":6126,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":63,"num_input_rowsets":2,"peak_mem_usage":569766,"rows_written":37700,"spinlock_wait_cycles":896,"thread_start_us":432,"threads_started":6}
I20240426 13:16:38.349180 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): free 1049059 bytes of WAL
I20240426 13:16:38.352535 10714 log_reader.cc:386] T e5d2e147c8c94844b14f3a93e5e47e6c: removed 1 log segments from log reader
I20240426 13:16:38.352649 10714 log.cc:1076] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833: Deleting log segment in path: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-0-root/wals/e5d2e147c8c94844b14f3a93e5e47e6c/wal-000000001 (ops 1-579)
I20240426 13:16:38.357199 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.005s	user 0.000s	sys 0.000s Metrics: {}
I20240426 13:16:38.357501 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 379406 bytes on disk
I20240426 13:16:38.357955 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20240426 13:16:38.358335 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:38.394707 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.365s	user 0.167s	sys 0.005s Metrics: {"bytes_written":271147,"cfile_init":1,"dirs.queue_time_us":29,"dirs.run_cpu_time_us":126,"dirs.run_wall_time_us":819,"drs_written":1,"lbm_read_time_us":45,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1127,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9400}
I20240426 13:16:38.395298 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 87574 bytes on disk
I20240426 13:16:38.400031 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4}
I20240426 13:16:38.400466 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014451
I20240426 13:16:38.776785 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.418s	user 0.233s	sys 0.011s Metrics: {"bytes_written":355844,"cfile_init":1,"dirs.queue_time_us":24,"dirs.run_cpu_time_us":153,"dirs.run_wall_time_us":850,"drs_written":1,"lbm_read_time_us":44,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3319,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":34,"peak_mem_usage":0,"rows_written":12300}
I20240426 13:16:38.777387 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014458
I20240426 13:16:38.933496 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.688s	user 0.652s	sys 0.008s Metrics: {"bytes_written":1332042,"cfile_cache_hit":13,"cfile_cache_hit_bytes":685840,"cfile_cache_miss":69,"cfile_cache_miss_bytes":2692380,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":9634,"dirs.run_cpu_time_us":128,"dirs.run_wall_time_us":5526,"drs_written":1,"lbm_read_time_us":2334,"lbm_reads_lt_1ms":89,"lbm_write_time_us":3974,"lbm_writes_lt_1ms":74,"num_input_rowsets":2,"peak_mem_usage":713098,"rows_written":47100,"spinlock_wait_cycles":384,"thread_start_us":322,"threads_started":6}
I20240426 13:16:38.934100 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 480761 bytes on disk
I20240426 13:16:38.934545 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":64,"lbm_reads_lt_1ms":4}
I20240426 13:16:39.251859 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=0.972434
I20240426 13:16:39.251916 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.851s	user 0.638s	sys 0.013s Metrics: {"bytes_written":1333205,"cfile_cache_hit":12,"cfile_cache_hit_bytes":577931,"cfile_cache_miss":70,"cfile_cache_miss_bytes":2804673,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":1109,"dirs.run_cpu_time_us":143,"dirs.run_wall_time_us":15340,"drs_written":1,"lbm_read_time_us":2266,"lbm_reads_lt_1ms":90,"lbm_write_time_us":3802,"lbm_writes_lt_1ms":74,"num_input_rowsets":2,"peak_mem_usage":713863,"rows_written":47150,"thread_start_us":115,"threads_started":1}
I20240426 13:16:39.254289 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 481357 bytes on disk
I20240426 13:16:39.255115 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":38,"lbm_reads_lt_1ms":4}
I20240426 13:16:39.268385 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:39.547533 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.279s	user 0.175s	sys 0.000s Metrics: {"bytes_written":275079,"cfile_init":1,"dirs.queue_time_us":29,"dirs.run_cpu_time_us":123,"dirs.run_wall_time_us":1639,"drs_written":1,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2933,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":31,"peak_mem_usage":0,"rows_written":9550,"spinlock_wait_cycles":768}
I20240426 13:16:39.548118 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 89438 bytes on disk
I20240426 13:16:39.549122 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20240426 13:16:39.549624 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014379
I20240426 13:16:39.618942 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.367s	user 0.173s	sys 0.008s Metrics: {"bytes_written":260527,"cfile_init":1,"dirs.queue_time_us":12649,"dirs.run_cpu_time_us":132,"dirs.run_wall_time_us":836,"drs_written":1,"lbm_read_time_us":56,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1391,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9000,"thread_start_us":85,"threads_started":1}
I20240426 13:16:39.619529 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014384
I20240426 13:16:39.776809 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.996s	user 0.664s	sys 0.016s Metrics: {"bytes_written":1414902,"cfile_cache_hit":11,"cfile_cache_hit_bytes":603981,"cfile_cache_miss":73,"cfile_cache_miss_bytes":2984935,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":9829,"dirs.run_cpu_time_us":109,"dirs.run_wall_time_us":1376,"drs_written":1,"lbm_read_time_us":2782,"lbm_reads_lt_1ms":97,"lbm_write_time_us":12919,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":77,"num_input_rowsets":2,"peak_mem_usage":757316,"rows_written":50000,"spinlock_wait_cycles":768,"thread_start_us":289,"threads_started":6}
I20240426 13:16:39.777267 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 513078 bytes on disk
I20240426 13:16:39.777581 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":43,"lbm_reads_lt_1ms":4}
I20240426 13:16:39.777891 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:39.980005 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.202s	user 0.181s	sys 0.000s Metrics: {"bytes_written":289633,"cfile_init":1,"dirs.queue_time_us":34,"dirs.run_cpu_time_us":119,"dirs.run_wall_time_us":881,"drs_written":1,"lbm_read_time_us":60,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1114,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":10100,"spinlock_wait_cycles":215424}
I20240426 13:16:39.980567 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): free 1051531 bytes of WAL
I20240426 13:16:39.981420 10714 log_reader.cc:386] T e5d2e147c8c94844b14f3a93e5e47e6c: removed 1 log segments from log reader
I20240426 13:16:39.981516 10714 log.cc:1076] T e5d2e147c8c94844b14f3a93e5e47e6c P 2c806d4334ed4b06a3222a351ecf0833: Deleting log segment in path: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-0-root/wals/e5d2e147c8c94844b14f3a93e5e47e6c/wal-000000002 (ops 580-1158)
I20240426 13:16:39.981909 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20240426 13:16:39.982208 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 93937 bytes on disk
I20240426 13:16:39.982542 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":49,"lbm_reads_lt_1ms":4}
I20240426 13:16:39.982926 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014139
I20240426 13:16:40.508479 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.959s	user 0.779s	sys 0.020s Metrics: {"bytes_written":1603512,"cfile_cache_hit":10,"cfile_cache_hit_bytes":442732,"cfile_cache_miss":82,"cfile_cache_miss_bytes":3633768,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":10784,"dirs.run_cpu_time_us":131,"dirs.run_wall_time_us":3831,"drs_written":1,"lbm_read_time_us":3267,"lbm_reads_lt_1ms":102,"lbm_write_time_us":4657,"lbm_writes_lt_1ms":86,"num_input_rowsets":2,"peak_mem_usage":859480,"rows_written":56700,"thread_start_us":339,"threads_started":6}
I20240426 13:16:40.509068 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 586708 bytes on disk
I20240426 13:16:40.509464 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":48,"lbm_reads_lt_1ms":4}
I20240426 13:16:40.540588 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.921s	user 0.731s	sys 0.016s Metrics: {"bytes_written":1584686,"cfile_cache_hit":10,"cfile_cache_hit_bytes":432142,"cfile_cache_miss":82,"cfile_cache_miss_bytes":3600968,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":1107,"dirs.run_cpu_time_us":102,"dirs.run_wall_time_us":2614,"drs_written":1,"lbm_read_time_us":2868,"lbm_reads_lt_1ms":106,"lbm_write_time_us":4360,"lbm_writes_lt_1ms":85,"num_input_rowsets":2,"peak_mem_usage":850330,"rows_written":56100,"thread_start_us":429,"threads_started":6}
I20240426 13:16:40.541105 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 579845 bytes on disk
I20240426 13:16:40.541507 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":52,"lbm_reads_lt_1ms":4}
I20240426 13:16:40.556792 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=0.972434
I20240426 13:16:40.624998 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:40.762073 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.205s	user 0.159s	sys 0.000s Metrics: {"bytes_written":236704,"cfile_init":1,"dirs.queue_time_us":96,"dirs.run_cpu_time_us":138,"dirs.run_wall_time_us":767,"drs_written":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1038,"lbm_writes_lt_1ms":30,"peak_mem_usage":0,"rows_written":8250}
I20240426 13:16:40.762668 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): free 1048257 bytes of WAL
I20240426 13:16:40.762907 10865 log_reader.cc:386] T e5d2e147c8c94844b14f3a93e5e47e6c: removed 1 log segments from log reader
I20240426 13:16:40.763006 10865 log.cc:1076] T e5d2e147c8c94844b14f3a93e5e47e6c P 1e6fd25eb2a64d94a05b6e057222181c: Deleting log segment in path: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-2-root/wals/e5d2e147c8c94844b14f3a93e5e47e6c/wal-000000002 (ops 596-1185)
I20240426 13:16:40.763454 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20240426 13:16:40.763723 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 76585 bytes on disk
I20240426 13:16:40.764077 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":53,"lbm_reads_lt_1ms":4}
I20240426 13:16:40.764532 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.013978
I20240426 13:16:41.021850 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.397s	user 0.184s	sys 0.004s Metrics: {"bytes_written":273802,"cfile_init":1,"dirs.queue_time_us":81,"dirs.run_cpu_time_us":133,"dirs.run_wall_time_us":1318,"drs_written":1,"lbm_read_time_us":51,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1123,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9500}
I20240426 13:16:41.022468 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): free 1049879 bytes of WAL
I20240426 13:16:41.022715 10790 log_reader.cc:386] T e5d2e147c8c94844b14f3a93e5e47e6c: removed 1 log segments from log reader
I20240426 13:16:41.022822 10790 log.cc:1076] T e5d2e147c8c94844b14f3a93e5e47e6c P 1d95e865423d4797aa98bed1e6fa621d: Deleting log segment in path: /tmp/dist-test-taskH_Ata7/test-tmp/txn_participant-itest.0.ShouldGCWals_ParticipantCopyITest.TestCopyParticipantOps_0.1714137102075184-398-0/minicluster-data/ts-1-root/wals/e5d2e147c8c94844b14f3a93e5e47e6c/wal-000000002 (ops 615-1208)
I20240426 13:16:41.025120 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: LogGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {}
I20240426 13:16:41.025353 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.013979
I20240426 13:16:41.186204 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 1.203s	user 0.807s	sys 0.020s Metrics: {"bytes_written":1697275,"cfile_cache_hit":10,"cfile_cache_hit_bytes":453362,"cfile_cache_miss":86,"cfile_cache_miss_bytes":3871612,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":12762,"dirs.run_cpu_time_us":114,"dirs.run_wall_time_us":10575,"drs_written":1,"lbm_read_time_us":3110,"lbm_reads_lt_1ms":106,"lbm_write_time_us":7867,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":88,"num_input_rowsets":2,"peak_mem_usage":911321,"rows_written":60100,"thread_start_us":444,"threads_started":6}
I20240426 13:16:41.186812 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=0.972434
I20240426 13:16:41.579602 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.393s	user 0.202s	sys 0.008s Metrics: {"bytes_written":263133,"cfile_init":1,"dirs.queue_time_us":38,"dirs.run_cpu_time_us":152,"dirs.run_wall_time_us":767,"drs_written":1,"lbm_read_time_us":57,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1390,"lbm_writes_lt_1ms":32,"peak_mem_usage":0,"rows_written":9100}
I20240426 13:16:41.580412 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 708445 bytes on disk
I20240426 13:16:41.581626 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":105,"lbm_reads_lt_1ms":8}
I20240426 13:16:41.585464 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014182
I20240426 13:16:42.209666 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 1.445s	user 0.986s	sys 0.008s Metrics: {"bytes_written":1831231,"cfile_cache_hit":13,"cfile_cache_hit_bytes":760876,"cfile_cache_miss":89,"cfile_cache_miss_bytes":3923292,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":12645,"dirs.run_cpu_time_us":131,"dirs.run_wall_time_us":839,"drs_written":1,"lbm_read_time_us":3872,"lbm_reads_lt_1ms":109,"lbm_write_time_us":6194,"lbm_writes_lt_1ms":94,"num_input_rowsets":2,"peak_mem_usage":985277,"rows_written":64950,"thread_start_us":367,"threads_started":6}
I20240426 13:16:42.211609 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:42.857888 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 1.832s	user 0.920s	sys 0.008s Metrics: {"bytes_written":1851831,"cfile_cache_hit":12,"cfile_cache_hit_bytes":628848,"cfile_cache_miss":92,"cfile_cache_miss_bytes":4102746,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":62311,"dirs.run_cpu_time_us":127,"dirs.run_wall_time_us":1085,"drs_written":1,"lbm_read_time_us":3354,"lbm_reads_lt_1ms":116,"lbm_write_time_us":18614,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":95,"num_input_rowsets":2,"peak_mem_usage":995185,"rows_written":65600,"thread_start_us":1233,"threads_started":1}
I20240426 13:16:42.860049 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:43.029850 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.812s	user 0.235s	sys 0.000s Metrics: {"bytes_written":358577,"cfile_init":1,"dirs.queue_time_us":19539,"dirs.run_cpu_time_us":130,"dirs.run_wall_time_us":1345,"drs_written":1,"lbm_read_time_us":51,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1315,"lbm_writes_lt_1ms":35,"peak_mem_usage":0,"rows_written":12400,"thread_start_us":124,"threads_started":1}
I20240426 13:16:43.030475 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c): 794954 bytes on disk
I20240426 13:16:43.031006 10865 maintenance_manager.cc:643] P 1e6fd25eb2a64d94a05b6e057222181c: UndoDeltaBlockGCOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":95,"lbm_reads_lt_1ms":8}
I20240426 13:16:43.031790 10925 maintenance_manager.cc:419] P 1e6fd25eb2a64d94a05b6e057222181c: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014177
I20240426 13:16:43.058573 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 1.473s	user 0.935s	sys 0.017s Metrics: {"bytes_written":1953606,"cfile_cache_hit":11,"cfile_cache_hit_bytes":601874,"cfile_cache_miss":95,"cfile_cache_miss_bytes":4388272,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":15447,"dirs.run_cpu_time_us":112,"dirs.run_wall_time_us":2201,"drs_written":1,"lbm_read_time_us":12225,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":113,"lbm_write_time_us":5498,"lbm_writes_lt_1ms":102,"num_input_rowsets":2,"peak_mem_usage":1050078,"rows_written":69200,"thread_start_us":412,"threads_started":6}
I20240426 13:16:43.059473 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.000000
I20240426 13:16:43.209378 10790 maintenance_manager.cc:643] P 1d95e865423d4797aa98bed1e6fa621d: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.348s	user 0.298s	sys 0.004s Metrics: {"bytes_written":503316,"cfile_init":1,"dirs.queue_time_us":31,"dirs.run_cpu_time_us":102,"dirs.run_wall_time_us":854,"drs_written":1,"lbm_read_time_us":50,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1663,"lbm_writes_lt_1ms":42,"peak_mem_usage":0,"rows_written":17600}
I20240426 13:16:43.210253 10850 maintenance_manager.cc:419] P 1d95e865423d4797aa98bed1e6fa621d: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014117
I20240426 13:16:43.355484   398 test_util.cc:268] Using random seed: -1224347304
I20240426 13:16:43.365543   398 test_workload.cc:401] TestWorkload: Skipping table creation because table test-workload already exists
F20240426 13:16:43.433171 11144 test_workload.cc:301] Already present: key already present
*** Check failure stack trace: ***
*** Aborted at 1714137403 (unix time) try "date -d @1714137403" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGABRT (@0x3e80000018e) received by PID 398 (TID 0x7f9ae5820700) from PID 398; stack trace: ***
    @     0x7f9b862e5980 (unknown) at ??:0
    @     0x7f9b83d33fb7 gsignal at ??:0
    @     0x7f9b83d35921 abort at ??:0
    @     0x7f9b84f3edbd google::LogMessage::Fail() at ??:0
    @     0x7f9b84f42b83 google::LogMessage::SendToLog() at ??:0
    @     0x7f9b84f3e7bc google::LogMessage::Flush() at ??:0
    @     0x7f9b84f3ff49 google::LogMessageFatal::~LogMessageFatal() at ??:0
    @     0x7f9b87ce1da6 kudu::TestWorkload::GetNumberOfErrors() at ??:0
    @     0x7f9b87ce0816 kudu::TestWorkload::WriteThread() at ??:0
I20240426 13:16:43.504072 10714 maintenance_manager.cc:643] P 2c806d4334ed4b06a3222a351ecf0833: FlushMRSOp(e5d2e147c8c94844b14f3a93e5e47e6c) complete. Timing: real 0.444s	user 0.285s	sys 0.004s Metrics: {"bytes_written":443825,"cfile_init":1,"dirs.queue_time_us":34,"dirs.run_cpu_time_us":144,"dirs.run_wall_time_us":1039,"drs_written":1,"lbm_read_time_us":42,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1668,"lbm_writes_lt_1ms":40,"peak_mem_usage":0,"rows_written":15400}
    @     0x7f9b87ceab7f std::__invoke_impl<>() at ??:0
I20240426 13:16:43.504599 10774 maintenance_manager.cc:419] P 2c806d4334ed4b06a3222a351ecf0833: Scheduling CompactRowSetsOp(e5d2e147c8c94844b14f3a93e5e47e6c): perf score=1.014044
    @     0x7f9b87cea426 std::__invoke<>() at ??:0
    @     0x7f9b87cebaf7 _ZNSt6thread8_InvokerISt5tupleIJMN4kudu12TestWorkloadEFvvEPS3_EEE9_M_invokeIJLm0ELm1EEEEDTcl8__invokespcl10_S_declvalIXT_EEEEESt12_Index_tupleIJXspT_EEE at ??:0
    @     0x7f9b87ceba5d std::thread::_Invoker<>::operator()() at ??:0
    @     0x7f9b87ceb9d2 std::thread::_State_impl<>::_M_run() at ??:0
    @     0x7f9b847596df (unknown) at ??:0
    @     0x7f9b862da6db start_thread at ??:0
    @     0x7f9b83e1671f clone at ??:0