I want a simple class that implements a fixed-size circular buffer. It should be efficient, easy on the eyes, generically typed.
For now it need not be MT-capable. I can always add a lock later, it won't be high-concurrency in any case.
Methods should be: .Add() and I guess .List(), where I retrieve all the entries. On second thought, Retrieval I think should be done via an indexer. At any moment I will want to be able to retrieve any element in the buffer by index. But keep in mind that from one moment to the next Element[n] may be different, as the circular buffer fills up and rolls over. This isn't a stack, it's a circular buffer.
Regarding "overflow": I would expect internally there would be an array holding the items, and over time the head and tail of the buffer will rotate around that fixed array. But that should be invisible from the user. There should be no externally-detectable "overflow" event or behavior.
This is not a school assignment - it is most commonly going to be used for a MRU cache or a fixed-size transaction or event log.
I would use an array of T, a head and tail pointer, and add and get methods.
Like: (Bug hunting is left to the user)
// Hijack these for simplicity
import java.nio.BufferOverflowException;
import java.nio.BufferUnderflowException;
public class CircularBuffer<T> {
private T[] buffer;
private int tail;
private int head;
#SuppressWarnings("unchecked")
public CircularBuffer(int n) {
buffer = (T[]) new Object[n];
tail = 0;
head = 0;
}
public void add(T toAdd) {
if (head != (tail - 1)) {
buffer[head++] = toAdd;
} else {
throw new BufferOverflowException();
}
head = head % buffer.length;
}
public T get() {
T t = null;
int adjTail = tail > head ? tail - buffer.length : tail;
if (adjTail < head) {
t = (T) buffer[tail++];
tail = tail % buffer.length;
} else {
throw new BufferUnderflowException();
}
return t;
}
public String toString() {
return "CircularBuffer(size=" + buffer.length + ", head=" + head + ", tail=" + tail + ")";
}
public static void main(String[] args) {
CircularBuffer<String> b = new CircularBuffer<String>(3);
for (int i = 0; i < 10; i++) {
System.out.println("Start: " + b);
b.add("One");
System.out.println("One: " + b);
b.add("Two");
System.out.println("Two: " + b);
System.out.println("Got '" + b.get() + "', now " + b);
b.add("Three");
System.out.println("Three: " + b);
// Test Overflow
// b.add("Four");
// System.out.println("Four: " + b);
System.out.println("Got '" + b.get() + "', now " + b);
System.out.println("Got '" + b.get() + "', now " + b);
// Test Underflow
// System.out.println("Got '" + b.get() + "', now " + b);
// Back to start, let's shift on one
b.add("Foo");
b.get();
}
}
}
This is how I would (or did) write an efficient circular buffer in Java. It's backed by a simple array. For my particular use case, I needed high concurrent throughput, so I used CAS for allocation of the index. I then created mechanisms for reliable copies including a CAS copy of the entire buffer. I used this in a case which is outlined in greater detail in short article.
import java.util.concurrent.atomic.AtomicLong;
import java.lang.reflect.Array;
/**
* A circular array buffer with a copy-and-swap cursor.
*
* <p>This class provides an list of T objects who's size is <em>unstable</em>.
* It's intended for capturing data where the frequency of sampling greatly
* outweighs the frequency of inspection (for instance, monitoring).</p>
*
* <p>This object keeps in memory a fixed size buffer which is used for
* capturing objects. It copies the objects to a snapshot array which may be
* worked with. The size of the snapshot array will vary based on the
* stability of the array during the copy operation.</p>
*
* <p>Adding buffer to the buffer is <em>O(1)</em>, and lockless. Taking a
* stable copy of the sample is <em>O(n)</em>.</p>
*/
public class ConcurrentCircularBuffer <T> {
private final AtomicLong cursor = new AtomicLong();
private final T[] buffer;
private final Class<T> type;
/**
* Create a new concurrent circular buffer.
*
* #param type The type of the array. This is captured for the same reason
* it's required by {#link java.util.List.toArray()}.
*
* #param bufferSize The size of the buffer.
*
* #throws IllegalArgumentException if the bufferSize is a non-positive
* value.
*/
public ConcurrentCircularBuffer (final Class <T> type,
final int bufferSize)
{
if (bufferSize < 1) {
throw new IllegalArgumentException(
"Buffer size must be a positive value"
);
}
this.type = type;
this.buffer = (T[]) new Object [ bufferSize ];
}
/**
* Add a new object to this buffer.
*
* <p>Add a new object to the cursor-point of the buffer.</p>
*
* #param sample The object to add.
*/
public void add (T sample) {
buffer[(int) (cursor.getAndIncrement() % buffer.length)] = sample;
}
/**
* Return a stable snapshot of the buffer.
*
* <p>Capture a stable snapshot of the buffer as an array. The snapshot
* may not be the same length as the buffer, any objects which were
* unstable during the copy will be factored out.</p>
*
* #return An array snapshot of the buffer.
*/
public T[] snapshot () {
T[] snapshots = (T[]) new Object [ buffer.length ];
/* Determine the size of the snapshot by the number of affected
* records. Trim the size of the snapshot by the number of records
* which are considered to be unstable during the copy (the amount the
* cursor may have moved while the copy took place).
*
* If the cursor eliminated the sample (if the sample size is so small
* compared to the rate of mutation that it did a full-wrap during the
* copy) then just treat the buffer as though the cursor is
* buffer.length - 1 and it was not changed during copy (this is
* unlikley, but it should typically provide fairly stable results).
*/
long before = cursor.get();
/* If the cursor hasn't yet moved, skip the copying and simply return a
* zero-length array.
*/
if (before == 0) {
return (T[]) Array.newInstance(type, 0);
}
System.arraycopy(buffer, 0, snapshots, 0, buffer.length);
long after = cursor.get();
int size = buffer.length - (int) (after - before);
long snapshotCursor = before - 1;
/* Highly unlikely, but the entire buffer was replaced while we
* waited...so just return a zero length array, since we can't get a
* stable snapshot...
*/
if (size <= 0) {
return (T[]) Array.newInstance(type, 0);
}
long start = snapshotCursor - (size - 1);
long end = snapshotCursor;
if (snapshotCursor < snapshots.length) {
size = (int) snapshotCursor + 1;
start = 0;
}
/* Copy the sample snapshot to a new array the size of our stable
* snapshot area.
*/
T[] result = (T[]) Array.newInstance(type, size);
int startOfCopy = (int) (start % snapshots.length);
int endOfCopy = (int) (end % snapshots.length);
/* If the buffer space wraps the physical end of the array, use two
* copies to construct the new array.
*/
if (startOfCopy > endOfCopy) {
System.arraycopy(snapshots, startOfCopy,
result, 0,
snapshots.length - startOfCopy);
System.arraycopy(snapshots, 0,
result, (snapshots.length - startOfCopy),
endOfCopy + 1);
}
else {
/* Otherwise it's a single continuous segment, copy the whole thing
* into the result.
*/
System.arraycopy(snapshots, startOfCopy,
result, 0, endOfCopy - startOfCopy + 1);
}
return (T[]) result;
}
/**
* Get a stable snapshot of the complete buffer.
*
* <p>This operation fetches a snapshot of the buffer using the algorithm
* defined in {#link snapshot()}. If there was concurrent modification of
* the buffer during the copy, however, it will retry until a full stable
* snapshot of the buffer was acquired.</p>
*
* <p><em>Note, for very busy buffers on large symmetric multiprocessing
* machines and supercomputers running data processing intensive
* applications, this operation has the potential of being fairly
* expensive. In practice on commodity hardware, dualcore processors and
* non-processing intensive systems (such as web services) it very rarely
* retries.</em></p>
*
* #return A full copy of the internal buffer.
*/
public T[] completeSnapshot () {
T[] snapshot = snapshot();
/* Try again until we get a snapshot that's the same size as the
* buffer... This is very often a single iteration, but it depends on
* how busy the system is.
*/
while (snapshot.length != buffer.length) {
snapshot = snapshot();
}
return snapshot;
}
/**
* The size of this buffer.
*/
public int size () {
return buffer.length;
}
}
I would use ArrayBlockingQueue or one of the other prebuilt Queue implementations, depending on what the needs are. Very rarely there is need to implement such a data structure yourself (unless it's a school assignment).
EDIT: Now that you have added the requirement "to retrieve any element in the buffer by index", I suppose that you need to implement your own class (unless google-collections or some other library provides one). A circular buffer is quite easy to implement, as JeeBee's example shows. You may also look at ArrayBlockingQueue's source code - its code is quite clean, just remove the locking and unneeded methods, and add methods for accessing it by index.
Use Java's ArrayDeque
Here is a ready-to-use CircularArrayList implementation for Java which I use in production code. By overriding AbstractList in the Java-recommended way, it supports all functionality you would expect from a standard List implementation in the Java Collections Framework (generic element type, subList, iteration etc.).
The following calls complete in O(1):
add(item) - adds at end of list
remove(0) - removes from beginning of list
get(i) - retrieves random element in list
Here is an implementation I've coded for my own use but that could be useful.
The buffer contains a maximum fixed set of items. The set is circular, old items are automatically removed. The caller can get items tail by an absolute incremental index (a long), but items may have been lost between calls too distant in time. This class is fully thread-safe.
public sealed class ConcurrentCircularBuffer<T> : ICollection<T>
{
private T[] _items;
private int _index;
private bool _full;
public ConcurrentCircularBuffer(int capacity)
{
if (capacity <= 1) // need at least two items
throw new ArgumentException(null, "capacity");
Capacity = capacity;
_items = new T[capacity];
}
public int Capacity { get; private set; }
public long TotalCount { get; private set; }
public int Count
{
get
{
lock (SyncObject) // full & _index need to be in sync
{
return _full ? Capacity : _index;
}
}
}
public void AddRange(IEnumerable<T> items)
{
if (items == null)
return;
lock (SyncObject)
{
foreach (var item in items)
{
AddWithLock(item);
}
}
}
private void AddWithLock(T item)
{
_items[_index] = item;
_index++;
if (_index == Capacity)
{
_full = true;
_index = 0;
}
TotalCount++;
}
public void Add(T item)
{
lock (SyncObject)
{
AddWithLock(item);
}
}
public void Clear()
{
lock (SyncObject)
{
_items = new T[Capacity];
_index = 0;
_full = false;
TotalCount = 0;
}
}
// this gives raw access to the underlying buffer. not sure I should keep that
public T this[int index]
{
get
{
return _items[index];
}
}
public T[] GetTail(long startIndex)
{
long lostCount;
return GetTail(startIndex, out lostCount);
}
public T[] GetTail(long startIndex, out long lostCount)
{
if (startIndex < 0 || startIndex >= TotalCount)
throw new ArgumentOutOfRangeException("startIndex");
T[] array = ToArray();
lostCount = (TotalCount - Count) - startIndex;
if (lostCount >= 0)
return array;
lostCount = 0;
// this maybe could optimized to not allocate the initial array
// but in multi-threading environment, I suppose this is arguable (and more difficult).
T[] chunk = new T[TotalCount - startIndex];
Array.Copy(array, array.Length - (TotalCount - startIndex), chunk, 0, chunk.Length);
return chunk;
}
public T[] ToArray()
{
lock (SyncObject)
{
T[] items = new T[_full ? Capacity : _index];
if (_full)
{
if (_index == 0)
{
Array.Copy(_items, items, Capacity);
}
else
{
Array.Copy(_items, _index, items, 0, Capacity - _index);
Array.Copy(_items, 0, items, Capacity - _index, _index);
}
}
else if (_index > 0)
{
Array.Copy(_items, items, _index);
}
return items;
}
}
public IEnumerator<T> GetEnumerator()
{
return ToArray().AsEnumerable().GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
bool ICollection<T>.Contains(T item)
{
return _items.Contains(item);
}
void ICollection<T>.CopyTo(T[] array, int arrayIndex)
{
if (array == null)
throw new ArgumentNullException("array");
if (array.Rank != 1)
throw new ArgumentException(null, "array");
if (arrayIndex < 0)
throw new ArgumentOutOfRangeException("arrayIndex");
if ((array.Length - arrayIndex) < Count)
throw new ArgumentException(null, "array");
T[] thisArray = ToArray();
Array.Copy(thisArray, 0, array, arrayIndex, thisArray.Length);
}
bool ICollection<T>.IsReadOnly
{
get
{
return false;
}
}
bool ICollection<T>.Remove(T item)
{
return false;
}
private static object _syncObject;
private static object SyncObject
{
get
{
if (_syncObject == null)
{
object obj = new object();
Interlocked.CompareExchange(ref _syncObject, obj, null);
}
return _syncObject;
}
}
}
Just use someone else's implementation:
The Power Collections Deque<T> is implemented by a circular buffer.
The power collections library is patchy but the Deque is perfectly acceptable expanding circular buffer.
Since you indicate that you do not want expansion and instead desire overwrite you could fairly easily modify the code to overwrite. This would simply involve removing the check for the pointers being logically adjacent and just writing anyway. At the same time the private buffer could be made readonly.
System.Collections.Generic.Queue - is simple circular buffer inside (T[] with head and tail, just like in sample from JeeBee).
In Guava 15, we introduced EvictingQueue, which is a non-blocking, bounded queue that automatically evicts (removes) elements from the head of the queue when attempting to add elements to a full queue. This is different from conventional bounded queues, which either block or reject new elements when full.
It sounds like this should suit your needs, and has a much simpler interface than using an ArrayDeque directly (it uses one under the hood though!).
More information can be found here.
I want to answer this question in the java perspective.
To implement a circular buffer with java, you probably need three things including: a circular buffer class, generic and few operations on it(In order to learn which operations you need and the inner mechanism in these operation, you might need to read wiki for circular buffer at first).
Secondly, the judgement of buffer full or empty should be treated very carefully.
Here I give two instinctive solutions for the full / empty judgement. In solution one, you needs to create two integer variants for storing both the current size of your buffer and the maximum size of your buffer. Obviously, if current size equals to the maximum size, the buffer is full.
In another solution, we set the last one storage place in idle (for circular buffer with size seven, we set the storage at seven in idle). According to this, we can determine the buffer is full when expression (rp+1)%MAXSIZE == fp; is satisfied.
For more clarification, here gives one implementations with the java language.
import java.nio.BufferOverflowException;
import java.nio.BufferUnderflowException;
public class CircularBuffer<T> {
private int front;
private int rear;
private int currentSize;
private int maxSize;
private T[] buffer;
public CircularBuffer(int n) {
buffer = (T[]) new Object[n];
front = 0;
rear = 0;
currentSize = 0;
maxSize = n;
}
public void push(T e) {
if (!isFull()) {
buffer[rear] = e;
currentSize++;
rear = (rear + 1) % maxSize;
} else throw new BufferOverflowException();
}
public T pop() {
if (!isEmpty()) {
T temp = buffer[front];
buffer[front] = null;
front = (front + 1) % maxSize;
currentSize--;
return temp;
} else throw new BufferUnderflowException();
}
public T peekFirst() {
if (!isEmpty()) {
return buffer[front];
} else return null;
}
public T peekLast() {
if (!isEmpty()) {
return buffer[rear - 1];
} else return null;
}
public int size() {
return currentSize;
}
public boolean isEmpty() {
if (currentSize == 0) {
return true;
} else return false;
}
public boolean isFull() {
if (currentSize == maxSize) {
return true;
} else return false;
}
public boolean clean() {
front = 0;
rear = 0;
while (rear != 0) {
buffer[rear] = null;
rear = (rear + 1) % maxSize;
}
return true;
}
public static void main(String[] args) {
CircularBuffer<Integer> buff = new CircularBuffer<>(7);
buff.push(0);
buff.push(1);
buff.push(2);
System.out.println(buff.size());
System.out.println("The head element is: " + buff.pop());
System.out.println("Size should be twoo: " + buff.size());
System.out.println("The last element is one: " + buff.peekLast());
System.out.println("Size should be two: " + buff.size());
buff.clean();
System.out.println("Size should be zero: " + buff.size());
}
}
if an lru cache would work, consider just using http://java.sun.com/javase/6/docs/api/java/util/LinkedHashMap.html#LinkedHashMap(int,%20float,%20boolean), http://java.sun.com/javase/6/docs/api/java/util/LinkedHashMap.html#removeEldestEntry(java.util.Map.Entry)
Here is another implementation which uses Apache common collection's BoundedFifoBuffer . please use CircularFifoQueue if you are using latest JAR from Apache as below class is deprecated
BoundedFifoBuffer apiCallHistory = new BoundedFifoBuffer(20);
for(int i =1 ; i < 25; i++){
if(apiCallHistory.isFull()){
System.out.println("removing :: "+apiCallHistory.remove());
}
apiCallHistory.add(i);
}
// The following is in C#
public class fqueue
{
// The following code implements a circular queue of objects
//private data:
private bool empty;
private bool full;
private int begin, end;
private object[] x;
//public data:
public fqueue()
{
empty = !(full = false);
begin = end = 0xA2;
x = new object[256];
return;
}
public fqueue(int size)
{
if (1 > size) throw new Exception("fqueue: Size cannot be zero or negative");
empty = !(full = false);
begin = end = 0xA2;
x = new object[size];
return;
}
public object write
{
set
{
if(full) throw new Exception("Write error: Queue is full");
end = empty ? end : (end + 1) % x.Length;
full = ((end + 1) % x.Length) == begin;
empty = false;
x[end] = value;
}
}
public object read
{
get
{
if(empty) throw new Exception("Read error: Queue is empty");
full = false;
object ret = x[begin];
begin = (empty=end==begin) ?
begin :
(begin + 1) % x.Length;
return ret;
}
}
public int maxSize
{
get
{
return x.Length;
}
}
public int queueSize
{
get
{
return end - begin + (empty ? 0 : 1 + ((end < begin) ? x.Length : 0));
}
}
public bool isEmpty
{
get
{
return empty;
}
}
public bool isFull
{
get
{
return full;
}
}
public int start
{
get
{
return begin;
}
}
public int finish
{
get
{
return end;
}
}
}
Related
My homework is about using Strategy Patterns and implementing Bubble sort, Quick sort, and Merge sort. I have done the first two but my Merge Sort seems to be giving me the wrong output when I input "bonakid" it gives me an output of "abkidon". May I know which part is wrong in the implementation?
Also, I have used this particular ISortStrategy on my Bubble Sort and Quick Sort so it would be nice if they would use the same one so I would not have to change the others.
class mergeSort : ISortStrategy
{
public string Sort(string input)
{
var result = "";
int size = (input.Length % 2 == 0) ? input.Length / 2 : (input.Length + 1) / 2;
if (input.Length > 1)
{
char[] left = input.Substring(0, input.Length / 2).ToCharArray();
char[] right = input.Substring(input.Length / 2, input.Length - (input.Length / 2)).ToCharArray();
// Sorting two halves recursively
Sort(left.Length.ToString());
Sort(right.Length.ToString());
//merging left and right sublists
result = merge(input, left, right);
}
return result;
}
public string merge(string result, char[] left, char[] right)
{
int i1 = 0; // indexing left
int i2 = 0; // indexing right
var theString = result;
var aStringBuilder = new StringBuilder(theString);
for (int i = 0; i < aStringBuilder.Length; i++)
{
if (i2 >= right.Length || (i1 < left.Length && left.GetValue(i1).ToString().CompareTo(right.GetValue(i2).ToString()) < 0))
{
aStringBuilder.Remove(i, 1);
aStringBuilder.Insert(i, left.GetValue(i1).ToString());
i1++;
}
else
{
aStringBuilder.Remove(i, 1);
aStringBuilder.Insert(i, right.GetValue(i2).ToString());
i2++;
}
}
theString = aStringBuilder.ToString();
return theString;
}
}
}
public interface ISortStrategy
{
string Sort(string input);
}
Here is a refined version that minimizes allocations and performs the merge with parallel execution. Taken from this question.
using System;
using System.Collections.Generic;
using System.Threading.Tasks;
public class Program
{
public static void Main()
{
ISortStrategy sort = new MergeSort();
Console.WriteLine(sort.Sort("bonakid"));
}
}
public interface ISortStrategy
{
string Sort(string input);
}
public class MergeSort : ISortStrategy
{
public string Sort(string input)
{
return SortAlgo.Merge(input).Result;
}
}
public static class SortAlgo
{
public async static ValueTask<string> Merge(
string input,
IComparer<char> comparer = null)
{
comparer ??= Comparer<char>.Default;
// buffer to contain the orginal data.
Memory<char> source = new char[input.Length];
input.AsMemory().CopyTo(source);
// other buffer, that starts as the output.
Memory<char> buffer = new char[source.Length];
// copy the stack allocated buffer out as the result;
return (await MergeLadder(
source,
buffer,
comparer)).ToString();
}
public async static ValueTask<T[]> Merge<T>(
T[] input,
IComparer<T> comparer = null)
{
comparer ??= Comparer<T>.Default;
// buffer to contain the orginal data.
Memory<T> source = new T[input.Length];
// intentionally copying input so it is not mutated.
input.AsMemory().CopyTo(source);
// other buffer, that starts as the output.
Memory<T> buffer = new T[source.Length];
// copy the stack allocated buffer out as the result;
return (await MergeLadder(
source,
buffer,
comparer)).ToArray();
}
/// <remarks>
/// Bottom up merge with alternatring buffers.
/// </remarks>
private async static ValueTask<ReadOnlyMemory<T>> MergeLadder<T>(
Memory<T> input,
Memory<T> output,
IComparer<T> comparer,
int width = 2)
{
var half = width / 2;
await WhenAll(GetMergeTasks(
input,
output,
comparer,
width,
half));
// Did the last stride cover the whole input?
if (width >= input.Length)
{
// Yes, everything is sorted
return output;
}
else
{
// No, walk the array again with double the width.
// Switch the buffers so we don't walk over the results.
return await MergeLadder(output, input, comparer, width * 2);
}
}
private async static IAsyncEnumerable<Task> GetMergeTasks<T>(
Memory<T> input,
Memory<T> output,
IComparer<T> comparer,
int width,
int half)
{
// walk the input sequence in mergable strides
// combine the left and right halves into the output
for (int i = 0; i < input.Length; i += width)
{
Memory<T> left;
Memory<T> right;
Memory<T> merge;
var remaining = input.Length - i;
if (remaining <= half)
{
// not enough left for a right.
left = input.Slice(i, remaining);
right = Memory<T>.Empty;
merge = output.Slice(i, remaining);
}
else if (remaining < width)
{
// not enought for a whole right.
left = input.Slice(i, half);
right = input.Slice(i + half, remaining - half);
merge = output.Slice(i, remaining);
}
else
{
// the full stride.
left = input.Slice(i, half);
right = input.Slice(i + half, half);
merge = output.Slice(i, width);
}
// Now merge the left and right for this stride.
yield return Task.Run(() => Merge(left, right, merge, comparer));
await Task.CompletedTask;
}
}
public static async ValueTask WhenAll(IAsyncEnumerable<Task> tasks)
{
ArgumentNullException.ThrowIfNull(tasks);
// We don't allocate the list if no task throws
List<Exception> exceptions = null;
await foreach(var task in tasks)
{
try
{
await task;
}
catch(Exception ex)
{
exceptions ??= new();
exceptions.Add(ex);
}
}
if (!(exceptions is null))
{
throw new AggregateException(exceptions);
}
}
private static void Merge<T>(
ReadOnlyMemory<T> leftMemory,
ReadOnlyMemory<T> rightMemory,
Memory<T> mergeMemory,
IComparer<T> comparer)
{
var left = leftMemory.Span;
var right = rightMemory.Span;
var merge = mergeMemory.Span;
//While either span has an element
for(int m = 0, l = 0, r= 0; l < left.Length || r < right.Length; m++)
{
if (l < left.Length && r < right.Length)
{
//both sides have elements
if (comparer.Compare(left[l], right[r]) <= 0)
{
// left is less than right
merge[m] = left[l];
l++;
}
else
{
// right is less than left
merge[m] = right[r];
r++;
}
}
else if (l < left.Length)
{
// only left has some left
merge[m] = left[l];
l++;
}
else
{
// only right has some left
merge[m] = right[r];
r++;
}
}
}
}
As demonstrated here, here is a working example,
using System;
using System.Linq;
using System.Collections.Generic;
public class Program
{
public static void Main()
{
ISortStrategy mergeSort = new MergeSort();
Console.WriteLine(mergeSort.Sort("bonakid"));
}
}
public interface ISortStrategy
{
string Sort(string input);
}
public class MergeSort : ISortStrategy
{
public string Sort(string input)
{
return new string(Homework.MergeSort(input).ToArray());
}
}
public static class Homework
{
public static IList<T> MergeSort<T>(
IEnumerable<T> inputSequence,
IComparer<T> comparer = null)
{
if (comparer == null)
{
comparer = Comparer<T>.Default;
}
IList<T> input = inputSequence.ToArray();
IList<T> left;
IList<T> right;
IList<T> result = new T[input.Count];
// As this is a recursive algorithm, we need to have a base case to
// avoid an infinite recursion and therfore a stack overflow.
if (input.Count <= 1)
{
return input;
}
// The exact midpoint of our array.
int midPoint = Math.DivRem(input.Count, 2, out int twoRemainder);
// Will represent our 'left' array.
left = new T[midPoint];
if (twoRemainder == 0)
{
//The array has an even number of elements, the left and right array
//will have the same number of elements.
right = new T[midPoint];
}
else
{
// The array has an odd number of elements, the right array
// will have one more element than left.
right = new T[midPoint + 1];
}
//populate left array
for (int i = 0; i < midPoint; i++)
{
left[i] = input[i];
}
// populate right array
// start index from the midpoint, as we have already populated
// the left array from 0 to midpont.
for (int x = 0, i = midPoint; i < input.Count; i++, x++)
{
right[x] = input[i];
}
//Recursively sort the left array
left = MergeSort(left);
//Recursively sort the right array
right = MergeSort(right);
//Merge our two sorted arrays
result = Merge(left, right, comparer);
return result;
}
//This method will be responsible for combining our two sorted arrays into one giant array
private static IList<T> Merge<T>(
IList<T> left,
IList<T> right,
IComparer<T> comparer)
{
int resultLength = right.Count + left.Count;
IList<T> result = new T[resultLength];
int indexLeft = 0, indexRight = 0, indexResult = 0;
//while either array still has an element
while (indexLeft < left.Count || indexRight < right.Count)
{
// if both arrays have elements
if (indexLeft < left.Count && indexRight < right.Count)
{
// If item on left array is less than item on right array, add that item to the result array
if (comparer.Compare(left[indexLeft], right[indexRight]) <= 0)
{
result[indexResult] = left[indexLeft];
indexLeft++;
indexResult++;
}
// else the item in the right array wll be added to the results array
else
{
result[indexResult] = right[indexRight];
indexRight++;
indexResult++;
}
}
//if only the left array still has elements, add all its items to the results array
else if (indexLeft < left.Count)
{
result[indexResult] = left[indexLeft];
indexLeft++;
indexResult++;
}
//if only the right array still has elements, add all its items to the results array
else if (indexRight < right.Count)
{
result[indexResult] = right[indexRight];
indexRight++;
indexResult++;
}
}
return result;
}
}
I'm out of ideas on this one. Tried originally myself and then copied from SO and google, which worked on all cases except one, however still didn't find a recursive algorithm that is fast enough for that particular test case in my assignment :/
In any case, why this:
public static int FindMaximum(int[] array)
{
if (array is null)
{
throw new ArgumentNullException(nameof(array));
}
if (array.Length == 0)
{
throw new ArgumentException(null);
}
return FindMaxRec(array, array.Length);
}
public static int FindMaxRec(int[] arr, int n)
{
if (n == 1)
{
return arr[0];
}
return Math.Max(arr[n - 1], FindMaxRec(arr, n - 1));
}
doesn't work with this TestCase?:
[Test]
[Order(0)]
[Timeout(5_000)]
public void FindMaximum_TestForLargeArray()
{
int expected = this.max;
int actual = FindMaximum(this.array);
Assert.AreEqual(expected, actual);
}
EDIT 1:
This works fine though, but I need recursive:
public static int FindMaximum(int[] array)
{
if (array is null)
{
throw new ArgumentNullException(nameof(array));
}
if (array.Length == 0)
{
throw new ArgumentException(null);
}
int maxValue = int.MinValue;
for (int i = 0; i < array.Length; i++)
{
if (array[i] > maxValue)
{
maxValue = array[i];
}
}
return maxValue;
}
You can try splitting array in two:
public static int FindMaximum(int[] array) {
if (null == array)
throw new ArgumentNullException(nameof(array));
if (array.Length <= 0)
throw new ArgumentException("Empty array is not allowed.", nameof(array));
return FindMaxRec(array, 0, array.Length - 1);
}
private static int FindMaxRec(int[] array, int from, int to) {
if (to < from)
throw new ArgumentOutOfRangeException(nameof(to));
if (to <= from + 1)
return Math.Max(array[from], array[to]);
return Math.Max(FindMaxRec(array, from, (from + to) / 2),
FindMaxRec(array, (from + to) / 2 + 1, to));
}
Demo:
Random random = new Random(123);
int[] data = Enumerable
.Range(0, 10_000_000)
.Select(_ => random.Next(1_000_000_000))
.ToArray();
Stopwatch sw = new Stopwatch();
sw.Start();
int max = FindMaximum(data);
sw.Stop();
Console.WriteLine($"max = {max}");
Console.WriteLine($"time = {sw.ElapsedMilliseconds}");
Outcome:
max = 999999635
time = 100
An easy way to turn a simple linear algorithm into a recursive one is to make use of the enumerator of the array.
public static int FindMax(int[] values)
{
using var enumerator = values.GetEnumerator();
return FindMaxRecursively(enumerator, int.MinValue);
}
private static T FindMaxRecursively<T>(IEnumerator<T> enumerator, T currentMax) where T : IComparable
{
if (!enumerator.MoveNext()) return currentMax;
var currentValue = enumerator.Current;
if (currentValue.CompareTo(currentMax) > 0) currentMax = currentValue;
return FindMaxRecursively(enumerator, currentMax);
}
This passes your test case and uses recursion.
Edit: Here is a more beginner friendly version of the above, with comments to explain what it is doing:
public static int FindMax(IEnumerable<int> values)
{
using var enumerator = values.GetEnumerator();//the using statement disposes the enumerator when we are done
//disposing the enumerator is important because we want to reset the index back to zero for the next time someone enumerates the array
return FindMaxRecursively(enumerator, int.MinValue);
}
private static int FindMaxRecursively(IEnumerator<int> enumerator, int currentMax)
{
if (!enumerator.MoveNext()) //move to the next item in the array. If there are no more items in the array MoveNext() returns false
return currentMax; //if there are no more items in the array return the current maximum value
var currentValue = enumerator.Current;//this is the value in the array at the current index
if (currentValue > currentMax) currentMax = currentValue;//if it's larger than the current maximum update the maximum
return FindMaxRecursively(enumerator, currentMax);//continue on to the next value, making sure to pass the current maximum
}
Something that might help understand this is that the IEnumerator is what enables foreach loops. Under the hood, foreach loops are just repeatedly calling MoveNext on an item that has an IEnumerator. Here is some more info on that topic.
public static int findMax(int[] a, int index) {
if (index > 0) {
return Math.max(a[index], findMax(a, index-1))
} else {
return a[0];
}
}
I'm dealing with c# concurrent-queue and multi-threading in socket-programming tcp/ip
First, I've already done with socket-programming itself. That means, I've already finished coding about client, server and stuffs about communication itself
basic structure is pipe-lined(producer-consumer problem) and now I'm doing with bit conversion
below is brief summary about my code
client-socket ->server-socket -> concurrent_queue_1(with type byte[65536],Thread_1 process this) -> concurrent_queue_2(with type double[40,3500], Thread_2 process this) -> display-data or other work(It can be gpu-work)
*(double[40,3500] can be changed to other size)
Till now,I've implemented putting_data into queue1(Thread1) and just dequeuing all(Thread2) and, its speed is about 700Mbps
The reason I used two concurrent_queue is, I want communication,and type conversion work to be processed in background regardless of main procedure about control things.
Here is the code about my own concurrent_queue with Blocking
public class BlockingConcurrentQueue<T> : IDisposable
{
private readonly ConcurrentQueue<T> _internalQueue;
private AutoResetEvent _autoResetEvent;
private long _consumed;
private long _isAddingCompleted = 0;
private long _produced;
private long _sleeping;
public BlockingConcurrentQueue()
{
_internalQueue = new ConcurrentQueue<T>();
_produced = 0;
_consumed = 0;
_sleeping = 0;
_autoResetEvent = new AutoResetEvent(false);
}
public bool IsAddingCompleted
{
get
{
return Interlocked.Read(ref _isAddingCompleted) == 1;
}
}
public bool IsCompleted
{
get
{
if (Interlocked.Read(ref _isAddingCompleted) == 1 && _internalQueue.IsEmpty)
return true;
else
return false;
}
}
public void CompleteAdding()
{
Interlocked.Exchange(ref _isAddingCompleted, 1);
}
public void Dispose()
{
_autoResetEvent.Dispose();
}
public void Enqueue(T item)
{
_internalQueue.Enqueue(item);
if (Interlocked.Read(ref _isAddingCompleted) == 1)
throw new InvalidOperationException("Adding Completed.");
Interlocked.Increment(ref _produced);
if (Interlocked.Read(ref _sleeping) == 1)
{
Interlocked.Exchange(ref _sleeping, 0);
_autoResetEvent.Set();
}
}
public bool TryDequeue(out T result)
{
if (Interlocked.Read(ref _consumed) == Interlocked.Read(ref _produced))
{
Interlocked.Exchange(ref _sleeping, 1);
_autoResetEvent.WaitOne();
}
if (_internalQueue.TryDequeue(out result))
{
Interlocked.Increment(ref _consumed);
return true;
}
return false;
}
}
My question is here
As I mentioned above, concurrent_queue1's type is byte[65536] and 65536 bytes = 8192 double data.
(40 * 3500=8192 * 17.08984375)
I want merge multiple 8192 double data into form of double[40,3500](size can be changed)and enqueue to concurrent_queue2 with Thread2
It's easy to do it with naive-approach(using many complex for loop) but it's slow cuz, It copys all the
data and expose to upper class or layer.
I'm searching method automatically enqueuing with matched size like foreach loop automatically iterates through 2D-array in row-major way, not yet found
Is there any fast way to merge 1D-byte array into form of 2D-double array and enqueue it?
Thanks for your help!
I try to understand your conversion rule, so I write this conversion code. Use Parallel to speed up the calculation.
int maxSize = 65536;
byte[] dim1Array = new byte[maxSize];
for (int i = 0; i < maxSize; ++i)
{
dim1Array[i] = byte.Parse((i % 256).ToString());
}
int dim2Row = 40;
int dim2Column = 3500;
int byteToDoubleRatio = 8;
int toDoubleSize = maxSize / byteToDoubleRatio;
double[,] dim2Array = new double[dim2Row, dim2Column];
Parallel.For(0, toDoubleSize, i =>
{
int row = i / dim2Column;
int col = i % dim2Column;
int originByteIndex = row * dim2Column * byteToDoubleRatio + col * byteToDoubleRatio;
dim2Array[row, col] = BitConverter.ToDouble(
dim1Array,
originByteIndex);
});
I have a number of objects each with 3 numerical properties: "high", "low" and "tiebreaker". They are to be sorted as such: if an object's low is higher than another object's high, it appears before it in the list. Likewise if an object's high is lower than another's low, it appears later in the list. But in the case that two objects have conflicting ranges (eg one's high is between the other object's low and high), the tiebreaker property is considered wherein the object with the higher tiebreaker value gets placed earlier on the list.
I am specifically working with c#, but I think the ideas here are language agnostic enough such that code of any sort (no puns) would be welcome.
Also, I have worked on this myself. I have a nested for-loop that is just not working out for me so far. I'd give up some code but I'm on my phone and that makes it a chore. Besides, this is probably a fun one for you and you don't need my ugly code in your way anyhow.
Are you assuming that Min <= Tie <= Max? You do not say so in your question, and if you do not, the sort order is not well defined because it is not transitive. For instance, writing your ranges as [Min, Tie, Max], consider:
A: [5,-10, 6]
B: [0, 1, 10]
C: [2, 3, 4]
A < B (because they overlap and -10 < 1)
B < C (because they overlap and 1 < 3)
but A > C (because they don't overlap and 5 > 4)
If they are you can define a custom IComparer<Range> for your Range class, and pass it to any c# sort method.
Update and here's one such implementation.
public struct RangeWithTie<T> where T : IEquatable<T>, IComparable<T>
{
readonly T min;
readonly T max;
readonly T tie;
readonly bool isNonEmpty;
public static Range<T> Empty = new Range<T>();
public static IComparer<RangeWithTie<T>> CreateSortingComparer()
{
return new RangeWithTieComparer();
}
public RangeWithTie(T start, T tie, T end)
{
// Enfore start <= tie <= end
var comparer = Comparer<T>.Default;
if (comparer.Compare(start, end) > 0) // if start > end
{
throw new ArgumentOutOfRangeException("start and end are reversed");
}
else if (comparer.Compare(start, tie) > 0)
{
throw new ArgumentOutOfRangeException("tie is less than start");
}
else if (comparer.Compare(tie, end) > 0)
{
throw new ArgumentOutOfRangeException("tie is bigger than end");
}
else
{
this.min = start;
this.max = end;
this.tie = tie;
}
this.isNonEmpty = true;
}
public T Min { get { return min; } }
public T Max { get { return max; } }
public T Tie { get { return tie; } }
public bool IsEmpty { get { return !isNonEmpty; } }
public class RangeWithTieComparer : IComparer<RangeWithTie<T>>
{
#region IComparer<RangeWithTie<T>> Members
public int Compare(RangeWithTie<T> x, RangeWithTie<T> y)
{
// return x - y.
if (x.IsEmpty)
{
if (y.IsEmpty)
return 0;
else
return -1;
}
else if (y.IsEmpty)
{
return 1;
}
var comparer = Comparer<T>.Default;
if (comparer.Compare(y.Min, x.Max) > 0)
return -1;
else if (comparer.Compare(x.Min, y.Max) > 0)
return 1;
return comparer.Compare(x.Tie, y.Tie);
}
#endregion
}
public override string ToString()
{
if (IsEmpty)
return "Empty";
StringBuilder s = new StringBuilder();
s.Append('[');
if (Min != null)
{
s.Append(Min.ToString());
}
s.Append(", ");
if (Tie != null)
{
s.Append(Tie.ToString());
}
s.Append(", ");
if (Max != null)
{
s.Append(Max.ToString());
}
s.Append(']');
return s.ToString();
}
}
This could be used like so:
var sortedRanges = ranges.OrderBy(x => x, RangeWithTie<double>.CreateSortingComparer()).ToArray();
I didn't make the struct implement IComparer<RangeWithTie<T>> directly because ranges with identical comparisons aren't necessarily equal. For instance, [-1,0,1] and [-2,0,1] have identical comparisons but are not equal.
A quick solution, and a console application to test it. This method will return the larger of two objects. Just replace dynamic with the appropriate object type you need.
class Program
{
private static object Sort(dynamic first, dynamic second)
{
if (OverlapExists(first, second))
{
// Note: If tiebreakers are equal, the first will be returned:
return first.tiebreaker >= second.tiebreaker ? first : second;
}
else
{
// Note: Only need to test one value (just high); Since we know
// there is no overlap, the whole object (both high and low) must
// be either over or under that which it is compared to:
return first.high > second.high ? first : second;
}
}
private static bool OverlapExists(dynamic first, dynamic second)
{
return (first.low < second.high) && (second.low < first.high);
}
static void Main(string[] args)
{
dynamic first = new {name="first", high = 10,
tiebreaker = 5, low = 1 };
dynamic second = new {name="second", high = 15,
tiebreaker = 12, low = 11 };
dynamic third = new {name="third", high = 20,
tiebreaker = 9, low = 6 };
var firstResult = Sort(first, second);
var secondResult = Sort(first, third);
var thirdResult = Sort(second, third);
Console.WriteLine("1) " + first.ToString()
+ "\nVS: " + second.ToString());
Console.WriteLine("Winner: " + firstResult.name);
Console.WriteLine("\n2) " + first.ToString()
+ "\nVS: " + third.ToString());
Console.WriteLine("Winner: " + secondResult.name);
Console.WriteLine("\n3) " + second.ToString()
+ "\nVS: " + third.ToString());
Console.WriteLine("Winner: " + thirdResult.name);
Console.ReadKey();
}
}
Let’s say you have a List<T> (T being your objects with High-, Low- and Tie- Property), then you can use
list.Sort(…);
with a Comparison<T> as a Parameter. That’s a delegate that takes 2 of you objects and should return < 0, when the first instance of your object should be a head of the other instance or 0 if they are of equal order (or > 0 if the second second object should be ahead of first).
Or you could pass an custom comparer (implementing IComparer<T>) which does basically the same as the Comparison<T> but inform of an interface.
No matter what your logic is, you may implement IComparable to enable an Array or List's sorting capability. So, as the follow code shows,
public class MyStuff : IComparable<MyStuff>
{
public int High { get; set; }
public int Low { get; set; }
public int TieBreaker { get; set; }
public int CompareTo(MyStuff other)
{
// if an object's low is higher than another object's high,
// it appears before it in the list
if ((this.Low > other.High) ||
// if its high is between the other object's low and
// high then compare their tiebreaker
(this.High > other.Low && this.High < other.High &&
this.TieBreaker > other.TieBreaker))
return 1;
else if (this.Low == other.High)
return 0;
else
return -1;
}
}
The basic idea is CompareTo returns either 1 (move this before other), 0 (retain both positions) or -1 (move this after other), depending on your ordering logic.
See IComparable<T>
class DataObject : IComparable<DataObject>
{
public double High, Low, Tiebreaker;
public int CompareTo(DataObject obj)
{
// this doesn't seem to make sense as a range sort, but seems to match your question...
// low > another high
if (this.Low != obj.High)
return this.Low.CompareTo(obj.High);
// otherwise sort tiebreaker ascending
else this.TieBreaker.CompareTo(obj.TieBreaker);
}
}
used as
var items = new[] { new DataObject(1,2,3), new DataObject(4,5,6) };
Array.Sort<DataObject>(items);
// items is now sorted
Given a collection, is there a way to get the last N elements of that collection? If there isn't a method in the framework, what would be the best way to write an extension method to do this?
collection.Skip(Math.Max(0, collection.Count() - N));
This approach preserves item order without a dependency on any sorting, and has broad compatibility across several LINQ providers.
It is important to take care not to call Skip with a negative number. Some providers, such as the Entity Framework, will produce an ArgumentException when presented with a negative argument. The call to Math.Max avoids this neatly.
The class below has all of the essentials for extension methods, which are: a static class, a static method, and use of the this keyword.
public static class MiscExtensions
{
// Ex: collection.TakeLast(5);
public static IEnumerable<T> TakeLast<T>(this IEnumerable<T> source, int N)
{
return source.Skip(Math.Max(0, source.Count() - N));
}
}
A brief note on performance:
Because the call to Count() can cause enumeration of certain data structures, this approach has the risk of causing two passes over the data. This isn't really a problem with most enumerables; in fact, optimizations exist already for Lists, Arrays, and even EF queries to evaluate the Count() operation in O(1) time.
If, however, you must use a forward-only enumerable and would like to avoid making two passes, consider a one-pass algorithm like Lasse V. Karlsen or Mark Byers describe. Both of these approaches use a temporary buffer to hold items while enumerating, which are yielded once the end of the collection is found.
coll.Reverse().Take(N).Reverse().ToList();
public static IEnumerable<T> TakeLast<T>(this IEnumerable<T> coll, int N)
{
return coll.Reverse().Take(N).Reverse();
}
UPDATE: To address clintp's problem: a) Using the TakeLast() method I defined above solves the problem, but if you really want the do it without the extra method, then you just have to recognize that while Enumerable.Reverse() can be used as an extension method, you aren't required to use it that way:
List<string> mystring = new List<string>() { "one", "two", "three" };
mystring = Enumerable.Reverse(mystring).Take(2).Reverse().ToList();
.NET Core 2.0+ provides the LINQ method TakeLast():
https://learn.microsoft.com/en-us/dotnet/api/system.linq.enumerable.takelast
example:
Enumerable
.Range(1, 10)
.TakeLast(3) // <--- takes last 3 items
.ToList()
.ForEach(i => System.Console.WriteLine(i))
// outputs:
// 8
// 9
// 10
Note: I missed your question title which said Using Linq, so my answer does not in fact use Linq.
If you want to avoid caching a non-lazy copy of the entire collection, you could write a simple method that does it using a linked list.
The following method will add each value it finds in the original collection into a linked list, and trim the linked list down to the number of items required. Since it keeps the linked list trimmed to this number of items the entire time through iterating through the collection, it will only keep a copy of at most N items from the original collection.
It does not require you to know the number of items in the original collection, nor iterate over it more than once.
Usage:
IEnumerable<int> sequence = Enumerable.Range(1, 10000);
IEnumerable<int> last10 = sequence.TakeLast(10);
...
Extension method:
public static class Extensions
{
public static IEnumerable<T> TakeLast<T>(this IEnumerable<T> collection,
int n)
{
if (collection == null)
throw new ArgumentNullException(nameof(collection));
if (n < 0)
throw new ArgumentOutOfRangeException(nameof(n), $"{nameof(n)} must be 0 or greater");
LinkedList<T> temp = new LinkedList<T>();
foreach (var value in collection)
{
temp.AddLast(value);
if (temp.Count > n)
temp.RemoveFirst();
}
return temp;
}
}
Here's a method that works on any enumerable but uses only O(N) temporary storage:
public static class TakeLastExtension
{
public static IEnumerable<T> TakeLast<T>(this IEnumerable<T> source, int takeCount)
{
if (source == null) { throw new ArgumentNullException("source"); }
if (takeCount < 0) { throw new ArgumentOutOfRangeException("takeCount", "must not be negative"); }
if (takeCount == 0) { yield break; }
T[] result = new T[takeCount];
int i = 0;
int sourceCount = 0;
foreach (T element in source)
{
result[i] = element;
i = (i + 1) % takeCount;
sourceCount++;
}
if (sourceCount < takeCount)
{
takeCount = sourceCount;
i = 0;
}
for (int j = 0; j < takeCount; ++j)
{
yield return result[(i + j) % takeCount];
}
}
}
Usage:
List<int> l = new List<int> {4, 6, 3, 6, 2, 5, 7};
List<int> lastElements = l.TakeLast(3).ToList();
It works by using a ring buffer of size N to store the elements as it sees them, overwriting old elements with new ones. When the end of the enumerable is reached the ring buffer contains the last N elements.
I am surprised that no one has mentioned it, but SkipWhile does have a method that uses the element's index.
public static IEnumerable<T> TakeLastN<T>(this IEnumerable<T> source, int n)
{
if (source == null)
throw new ArgumentNullException("Source cannot be null");
int goldenIndex = source.Count() - n;
return source.SkipWhile((val, index) => index < goldenIndex);
}
//Or if you like them one-liners (in the spirit of the current accepted answer);
//However, this is most likely impractical due to the repeated calculations
collection.SkipWhile((val, index) => index < collection.Count() - N)
The only perceivable benefit that this solution presents over others is that you can have the option to add in a predicate to make a more powerful and efficient LINQ query, instead of having two separate operations that traverse the IEnumerable twice.
public static IEnumerable<T> FilterLastN<T>(this IEnumerable<T> source, int n, Predicate<T> pred)
{
int goldenIndex = source.Count() - n;
return source.SkipWhile((val, index) => index < goldenIndex && pred(val));
}
Use EnumerableEx.TakeLast in RX's System.Interactive assembly. It's an O(N) implementation like #Mark's, but it uses a queue rather than a ring-buffer construct (and dequeues items when it reaches buffer capacity).
(NB: This is the IEnumerable version - not the IObservable version, though the implementation of the two is pretty much identical)
If you are dealing with a collection with a key (e.g. entries from a database) a quick (i.e. faster than the selected answer) solution would be
collection.OrderByDescending(c => c.Key).Take(3).OrderBy(c => c.Key);
If you don't mind dipping into Rx as part of the monad, you can use TakeLast:
IEnumerable<int> source = Enumerable.Range(1, 10000);
IEnumerable<int> lastThree = source.AsObservable().TakeLast(3).AsEnumerable();
I tried to combine efficiency and simplicity and end up with this :
public static IEnumerable<T> TakeLast<T>(this IEnumerable<T> source, int count)
{
if (source == null) { throw new ArgumentNullException("source"); }
Queue<T> lastElements = new Queue<T>();
foreach (T element in source)
{
lastElements.Enqueue(element);
if (lastElements.Count > count)
{
lastElements.Dequeue();
}
}
return lastElements;
}
About
performance : In C#, Queue<T> is implemented using a circular buffer so there is no object instantiation done each loop (only when the queue is growing up). I did not set queue capacity (using dedicated constructor) because someone might call this extension with count = int.MaxValue . For extra performance you might check if source implement IList<T> and if yes, directly extract the last values using array indexes.
If using a third-party library is an option, MoreLinq defines TakeLast() which does exactly this.
It is a little inefficient to take the last N of a collection using LINQ as all the above solutions require iterating across the collection. TakeLast(int n) in System.Interactive also has this problem.
If you have a list a more efficient thing to do is slice it using the following method
/// Select from start to end exclusive of end using the same semantics
/// as python slice.
/// <param name="list"> the list to slice</param>
/// <param name="start">The starting index</param>
/// <param name="end">The ending index. The result does not include this index</param>
public static List<T> Slice<T>
(this IReadOnlyList<T> list, int start, int? end = null)
{
if (end == null)
{
end = list.Count();
}
if (start < 0)
{
start = list.Count + start;
}
if (start >= 0 && end.Value > 0 && end.Value > start)
{
return list.GetRange(start, end.Value - start);
}
if (end < 0)
{
return list.GetRange(start, (list.Count() + end.Value) - start);
}
if (end == start)
{
return new List<T>();
}
throw new IndexOutOfRangeException(
"count = " + list.Count() +
" start = " + start +
" end = " + end);
}
with
public static List<T> GetRange<T>( this IReadOnlyList<T> list, int index, int count )
{
List<T> r = new List<T>(count);
for ( int i = 0; i < count; i++ )
{
int j=i + index;
if ( j >= list.Count )
{
break;
}
r.Add(list[j]);
}
return r;
}
and some test cases
[Fact]
public void GetRange()
{
IReadOnlyList<int> l = new List<int>() { 0, 10, 20, 30, 40, 50, 60 };
l
.GetRange(2, 3)
.ShouldAllBeEquivalentTo(new[] { 20, 30, 40 });
l
.GetRange(5, 10)
.ShouldAllBeEquivalentTo(new[] { 50, 60 });
}
[Fact]
void SliceMethodShouldWork()
{
var list = new List<int>() { 1, 3, 5, 7, 9, 11 };
list.Slice(1, 4).ShouldBeEquivalentTo(new[] { 3, 5, 7 });
list.Slice(1, -2).ShouldBeEquivalentTo(new[] { 3, 5, 7 });
list.Slice(1, null).ShouldBeEquivalentTo(new[] { 3, 5, 7, 9, 11 });
list.Slice(-2)
.Should()
.BeEquivalentTo(new[] {9, 11});
list.Slice(-2,-1 )
.Should()
.BeEquivalentTo(new[] {9});
}
I know it's to late to answer this question. But if you are working with collection of type IList<> and you don't care about an order of the returned collection, then this method is working faster. I've used Mark Byers answer and made a little changes. So now method TakeLast is:
public static IEnumerable<T> TakeLast<T>(IList<T> source, int takeCount)
{
if (source == null) { throw new ArgumentNullException("source"); }
if (takeCount < 0) { throw new ArgumentOutOfRangeException("takeCount", "must not be negative"); }
if (takeCount == 0) { yield break; }
if (source.Count > takeCount)
{
for (int z = source.Count - 1; takeCount > 0; z--)
{
takeCount--;
yield return source[z];
}
}
else
{
for(int i = 0; i < source.Count; i++)
{
yield return source[i];
}
}
}
For test I have used Mark Byers method and kbrimington's andswer. This is test:
IList<int> test = new List<int>();
for(int i = 0; i<1000000; i++)
{
test.Add(i);
}
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
IList<int> result = TakeLast(test, 10).ToList();
stopwatch.Stop();
Stopwatch stopwatch1 = new Stopwatch();
stopwatch1.Start();
IList<int> result1 = TakeLast2(test, 10).ToList();
stopwatch1.Stop();
Stopwatch stopwatch2 = new Stopwatch();
stopwatch2.Start();
IList<int> result2 = test.Skip(Math.Max(0, test.Count - 10)).Take(10).ToList();
stopwatch2.Stop();
And here are results for taking 10 elements:
and for taking 1000001 elements results are:
Here's my solution:
public static class EnumerationExtensions
{
public static IEnumerable<T> TakeLast<T>(this IEnumerable<T> input, int count)
{
if (count <= 0)
yield break;
var inputList = input as IList<T>;
if (inputList != null)
{
int last = inputList.Count;
int first = last - count;
if (first < 0)
first = 0;
for (int i = first; i < last; i++)
yield return inputList[i];
}
else
{
// Use a ring buffer. We have to enumerate the input, and we don't know in advance how many elements it will contain.
T[] buffer = new T[count];
int index = 0;
count = 0;
foreach (T item in input)
{
buffer[index] = item;
index = (index + 1) % buffer.Length;
count++;
}
// The index variable now points at the next buffer entry that would be filled. If the buffer isn't completely
// full, then there are 'count' elements preceding index. If the buffer *is* full, then index is pointing at
// the oldest entry, which is the first one to return.
//
// If the buffer isn't full, which means that the enumeration has fewer than 'count' elements, we'll fix up
// 'index' to point at the first entry to return. That's easy to do; if the buffer isn't full, then the oldest
// entry is the first one. :-)
//
// We'll also set 'count' to the number of elements to be returned. It only needs adjustment if we've wrapped
// past the end of the buffer and have enumerated more than the original count value.
if (count < buffer.Length)
index = 0;
else
count = buffer.Length;
// Return the values in the correct order.
while (count > 0)
{
yield return buffer[index];
index = (index + 1) % buffer.Length;
count--;
}
}
}
public static IEnumerable<T> SkipLast<T>(this IEnumerable<T> input, int count)
{
if (count <= 0)
return input;
else
return input.SkipLastIter(count);
}
private static IEnumerable<T> SkipLastIter<T>(this IEnumerable<T> input, int count)
{
var inputList = input as IList<T>;
if (inputList != null)
{
int first = 0;
int last = inputList.Count - count;
if (last < 0)
last = 0;
for (int i = first; i < last; i++)
yield return inputList[i];
}
else
{
// Aim to leave 'count' items in the queue. If the input has fewer than 'count'
// items, then the queue won't ever fill and we return nothing.
Queue<T> elements = new Queue<T>();
foreach (T item in input)
{
elements.Enqueue(item);
if (elements.Count > count)
yield return elements.Dequeue();
}
}
}
}
The code is a bit chunky, but as a drop-in reusable component, it should perform as well as it can in most scenarios, and it'll keep the code that's using it nice and concise. :-)
My TakeLast for non-IList`1 is based on the same ring buffer algorithm as that in the answers by #Mark Byers and #MackieChan further up. It's interesting how similar they are -- I wrote mine completely independently. Guess there's really just one way to do a ring buffer properly. :-)
Looking at #kbrimington's answer, an additional check could be added to this for IQuerable<T> to fall back to the approach that works well with Entity Framework -- assuming that what I have at this point does not.
Honestly I'm not super proud of the answer, but for small collections you could use the following:
var lastN = collection.Reverse().Take(n).Reverse();
A bit hacky but it does the job ;)
My solution is based on ranges, introduced in C# version 8.
public static IEnumerable<T> TakeLast<T>(this IEnumerable<T> source, int N)
{
return source.ToArray()[(source.Count()-N)..];
}
After running a benchmark with most rated solutions (and my humbly proposed solution):
public static class TakeLastExtension
{
public static IEnumerable<T> TakeLastMarkByers<T>(this IEnumerable<T> source, int takeCount)
{
if (source == null) { throw new ArgumentNullException("source"); }
if (takeCount < 0) { throw new ArgumentOutOfRangeException("takeCount", "must not be negative"); }
if (takeCount == 0) { yield break; }
T[] result = new T[takeCount];
int i = 0;
int sourceCount = 0;
foreach (T element in source)
{
result[i] = element;
i = (i + 1) % takeCount;
sourceCount++;
}
if (sourceCount < takeCount)
{
takeCount = sourceCount;
i = 0;
}
for (int j = 0; j < takeCount; ++j)
{
yield return result[(i + j) % takeCount];
}
}
public static IEnumerable<T> TakeLastKbrimington<T>(this IEnumerable<T> source, int N)
{
return source.Skip(Math.Max(0, source.Count() - N));
}
public static IEnumerable<T> TakeLastJamesCurran<T>(this IEnumerable<T> source, int N)
{
return source.Reverse().Take(N).Reverse();
}
public static IEnumerable<T> TakeLastAlex<T>(this IEnumerable<T> source, int N)
{
return source.ToArray()[(source.Count()-N)..];
}
}
Test
[MemoryDiagnoser]
public class TakeLastBenchmark
{
[Params(10000)]
public int N;
private readonly List<string> l = new();
[GlobalSetup]
public void Setup()
{
for (var i = 0; i < this.N; i++)
{
this.l.Add($"i");
}
}
[Benchmark]
public void Benchmark1_MarkByers()
{
var lastElements = l.TakeLastMarkByers(3).ToList();
}
[Benchmark]
public void Benchmark2_Kbrimington()
{
var lastElements = l.TakeLastKbrimington(3).ToList();
}
[Benchmark]
public void Benchmark3_JamesCurran()
{
var lastElements = l.TakeLastJamesCurran(3).ToList();
}
[Benchmark]
public void Benchmark4_Alex()
{
var lastElements = l.TakeLastAlex(3).ToList();
}
}
Program.cs:
var summary = BenchmarkRunner.Run(typeof(TakeLastBenchmark).Assembly);
Command dotnet run --project .\TestsConsole2.csproj -c Release --logBuildOutput
The results were following:
// * Summary *
BenchmarkDotNet=v0.13.2, OS=Windows 10 (10.0.19044.1889/21H2/November2021Update)
AMD Ryzen 5 5600X, 1 CPU, 12 logical and 6 physical cores
.NET SDK=6.0.401
[Host] : .NET 6.0.9 (6.0.922.41905), X64 RyuJIT AVX2
DefaultJob : .NET 6.0.9 (6.0.922.41905), X64 RyuJIT AVX2
Method
N
Mean
Error
StdDev
Gen0
Gen1
Allocated
Benchmark1_MarkByers
10000
89,390.53 ns
1,735.464 ns
1,704.457 ns
-
-
248 B
Benchmark2_Kbrimington
10000
46.15 ns
0.410 ns
0.363 ns
0.0076
-
128 B
Benchmark3_JamesCurran
10000
2,703.15 ns
46.298 ns
67.862 ns
4.7836
0.0038
80264 B
Benchmark4_Alex
10000
2,513.48 ns
48.661 ns
45.517 ns
4.7607
-
80152 B
Turns out the solution proposed by #Kbrimington to be the most efficient in terms of memory alloc as well as raw performance.
Below the real example how to take last 3 elements from a collection (array):
// split address by spaces into array
string[] adrParts = adr.Split(new string[] { " " },StringSplitOptions.RemoveEmptyEntries);
// take only 3 last items in array
adrParts = adrParts.SkipWhile((value, index) => { return adrParts.Length - index > 3; }).ToArray();
Using This Method To Get All Range Without Error
public List<T> GetTsRate( List<T> AllT,int Index,int Count)
{
List<T> Ts = null;
try
{
Ts = AllT.ToList().GetRange(Index, Count);
}
catch (Exception ex)
{
Ts = AllT.Skip(Index).ToList();
}
return Ts ;
}
Little different implementation with usage of circular buffer. The benchmarks show that the method is circa two times faster than ones using Queue (implementation of TakeLast in System.Linq), however not without a cost - it needs a buffer which grows along with the requested number of elements, even if you have a small collection you can get huge memory allocation.
public IEnumerable<T> TakeLast<T>(IEnumerable<T> source, int count)
{
int i = 0;
if (count < 1)
yield break;
if (source is IList<T> listSource)
{
if (listSource.Count < 1)
yield break;
for (i = listSource.Count < count ? 0 : listSource.Count - count; i < listSource.Count; i++)
yield return listSource[i];
}
else
{
bool move = true;
bool filled = false;
T[] result = new T[count];
using (var enumerator = source.GetEnumerator())
while (move)
{
for (i = 0; (move = enumerator.MoveNext()) && i < count; i++)
result[i] = enumerator.Current;
filled |= move;
}
if (filled)
for (int j = i; j < count; j++)
yield return result[j];
for (int j = 0; j < i; j++)
yield return result[j];
}
}
//detailed code for the problem
//suppose we have a enumerable collection 'collection'
var lastIndexOfCollection=collection.Count-1 ;
var nthIndexFromLast= lastIndexOfCollection- N;
var desiredCollection=collection.GetRange(nthIndexFromLast, N);
---------------------------------------------------------------------
// use this one liner
var desiredCollection=collection.GetRange((collection.Count-(1+N)), N);