1, Foreword

The author is working on a project recently. In order to improve the throughput, the message queue is used to realize the production and consumption mode. In the production and consumption mode, there needs to be a collection to store the items produced by the producer. The author uses the most common list < T > collection type.

Because there are many producer threads and many consumer threads, the problem of thread synchronization is inevitable. At first, the author used the lock keyword to synchronize threads, but the performance was not particularly ideal. Then some netizens said that synchronized list < T > could be used instead of list < T > to achieve thread safety. Therefore, the author replaced it with synchronized list < T >, but found that the performance is still poor. Therefore, I checked the source code of synchronized list < T >, and found that it simply added lock on the basis of the API provided by list < T >, so the performance is basically the same as the author's implementation method.

Finally, the author found a solution and implemented it with the concurrentbag < T > class, which has greatly improved the performance. Therefore, the author checked the source code of concurrentbag < T > and the implementation is very exquisite. I hereby record it here.

2, ConcurrentBag class

Concurrentbag < T > implements the iproducerconsumercollection < T > interface, which is mainly used in the producer consumer mode. It can be seen that this class is basically customized for the producer consumer mode. Then it also implements the conventional ireadonlycollection < T > class. To implement this class, you need to implement IEnumerable < T >, IEnumerable and ICollection classes.

Concurrentbag < T > does not provide as many external methods as list < T >, but it also has the extension method implemented by Enumerable. The methods provided by the class itself are as follows.

3, Implementation principle of ConcurrentBag thread safety

1. Private field of concurrentbag

Concurrent bag thread safety is mainly realized through its data storage structure and fine-grained locks.

 public class ConcurrentBag<T> : IProducerConsumerCollection<T>, IReadOnlyCollection<T>
    {        // The ThreadLocalList object contains data for each thread
        ThreadLocal<ThreadLocalList> m_locals; 
        // This header pointer and tail pointer point to the first and last local lists in the, which are scattered across different threads
        // Allow enumeration on thread local objects
        volatile ThreadLocalList m_headList, m_tailList; 
        // This flag tells the operating thread that it must synchronize operations
        // Set in GlobalListsLock lock
        bool m_needSync;

}

Let's first look at the private field declared by it. It should be noted that the data of the collection is stored in the local storage of ThreadLocal thread. In other words, each thread accessing it will maintain its own collection data list, and the data in a collection may be stored in the local storage space of different threads. Therefore, if a thread accesses its own locally stored objects, there is no problem. This is the first layer of thread safety, using threads to store data locally.

Then you can see ThreadLocalList m_headList, m_tailList; This is the head pointer and tail pointer of the local list object. Through these two pointers, we can access all local lists through traversal. It uses volatile decoration, so it is thread safe.

Finally, a flag is defined, which tells the operating thread that synchronization operation must be performed. This is to realize a fine-grained lock, because thread synchronization is required only when several conditions are met.

2. TrehadLocalList class for data storage

Next, let's take a look at the construction of the ThreadLocalList class, which is the location where the data is actually stored. In fact, it uses the structure of two-way linked list for data storage.

[Serializable]// The node internal class node {public node (t value) of the bidirectional linked list is constructed{
        m_value = value;
    }    public readonly T m_value;    public Node m_next;    public Node m_prev;
}///< summary > / / / collection operation type / / < / summary > internal enum listoperation
{
    None,
    Add,
    Take
};///< summary > / / / thread locked classes / / < / summary > internal class threadlocallist {/ / if the head node of the two-way linked list is null, the linked list is empty
    internal volatile Node m_head;    // Tail node of bidirectional linked list
    private volatile Node m_tail;    // Defines the type of operation currently performed on the List 
    // Corresponds to the previous ListOperation
    internal volatile int m_currentOp;    // The count of this list element
    private int m_count;    // The stealing count
    // This is not particularly understood. It seems to be the count after deleting a Node in the local list
    internal int m_stealCount;    // The next list may be in another thread
    internal volatile ThreadLocalList m_nextList;    // Set whether locking has been performed
    internal bool m_lockTaken;    // The owner thread for this list
    internal Thread m_ownerThread;    // The version of the list. Only when the list changes from empty to non empty is the underlying statistics
    internal volatile int m_version;    /// <summary>
    ///ThreadLocalList constructor
    /// </summary>
    ///< param name = "ownerthread" > the thread that owns this collection < / param >
    internal ThreadLocalList(Thread ownerThread)    {
        m_ownerThread = ownerThread;
    }    /// <summary>
    ///Add a new item to the head of the linked list
    /// </summary>
    /// <param name="item">The item to add.</param>
    ///< param name = "updatecount" > update count. < / param >
    internal void Add(T item, bool updateCount)    {        checked
        {
            m_count++;
        }
        Node node = new Node(item);        if (m_head == null)
        {
            Debug.Assert(m_tail == null);
            m_head = node;
            m_tail = node;
            m_version++; // Because of initialization, the empty state is changed to a non empty state
        }        else
        {            // Insert a new element into the linked list using header insertion
            node.m_next = m_head;
            m_head.m_prev = node;
            m_head = node;
        }        if (updateCount) // Update the count to avoid overflow during this add synchronization
        {
            m_count = m_count - m_stealCount;
            m_stealCount = 0;
        }
    }    /// <summary>
    ///Delete an item from the head of the list
    /// </summary>
    /// <param name="result">The removed item</param>
    internal void Remove(out T result)    {        // Process of deleting header node data from bidirectional linked list
        Debug.Assert(m_head != null);
        Node head = m_head;
        m_head = m_head.m_next;        if (m_head != null)
        {
            m_head.m_prev = null;
        }        else
        {
            m_tail = null;
        }
        m_count--;
        result = head.m_value;

    }    /// <summary>
    ///Returns the element at the head of the list
    /// </summary>
    /// <param name="result">the peeked item</param>
    /// <returns>True if succeeded, false otherwise</returns>
    internal bool Peek(out T result)    {
        Node head = m_head;        if (head != null)
        {
            result = head.m_value;            return true;
        }
        result = default(T);        return false;
    }    /// <summary>
    ///Get an item from the end of the list
    /// </summary>
    /// <param name="result">the removed item</param>
    /// <param name="remove">remove or peek flag</param>
    internal void Steal(out T result, bool remove)    {
        Node tail = m_tail;
        Debug.Assert(tail != null);        if (remove) // Take operation
        {
            m_tail = m_tail.m_prev;            if (m_tail != null)
            {
                m_tail.m_next = null;
            }            else
            {
                m_head = null;
            }            // Increment the steal count
            m_stealCount++;
        }
        result = tail.m_value;
    }    /// <summary>
    ///Gets the total list count, which is not thread safe. If it is called at the same time, it may provide an incorrect count
    /// </summary>
    internal int Count
    {        get
        {            return m_count - m_stealCount;
        }
    }
}

From the above code, we can further verify the previous view that when concurrentbag < T > stores data in a thread, it uses a two-way linked list, and ThreadLocalList implements a set of methods to add, delete, modify and query the linked list.

3. ConcurrentBag implements new elements

Next, let's take a look at how concurrentbag < T > adds new elements.

///< summary > / / / try to get the unowned list. The unowned list means that the thread has been suspended or terminated, but some data in the collection is still stored there / / this is a way to avoid memory leakage / / < / summary > / / < returns > < / returns > private threadlocallist getunownedlist() {/ / a global lock must be held at this time
    Contract.Assert(Monitor.IsEntered(GlobalListsLock));    // Start enumerating from the thread list to find those threads that have been closed
    // Returns the list object it is in
    ThreadLocalList currentList = m_headList;    while (currentList != null)
    {        if (currentList.m_ownerThread.ThreadState == System.Threading.ThreadState.Stopped)
        {
            currentList.m_ownerThread = Thread.CurrentThread; // the caller should acquire a lock to make this line thread safe
            return currentList;
        }
        currentList = currentList.m_nextList;
    }    return null;
}///< summary > / / / local help method to retrieve the thread local list through the thread object / / < / summary > / / < param name = "forcecreate" > if the list does not exist, create a new list < / param > / / < returns > the local list object < / returns > private threadlocallist getthreadlist (bool forcecreate){
    ThreadLocalList list = m_locals.Value;    if (list != null)
    {        return list;
    }    else if (forceCreate)    {        // Gets the m_tailList lock used for the update operation
        lock (GlobalListsLock)
        {            // If the header list is empty, there are no elements in the collection
            // Directly create a new
            if (m_headList == null)
            {
                list = new ThreadLocalList(Thread.CurrentThread);
                m_headList = list;
                m_tailList = list;
            }            else
            {               // The data in ConcurrentBag is distributed and stored in the local area of each thread in the form of two-way linked list
                // Through the following method, you can find the threads that have stored data but have been stopped
                // Then hand over the data of the stopped thread to the current thread management
                list = GetUnownedList();                // If not, create a new list and update the position of the tail pointer
                if (list == null)
                {
                    list = new ThreadLocalList(Thread.CurrentThread);
                    m_tailList.m_nextList = list;
                    m_tailList = list;
                }
            }
            m_locals.Value = list;
        }
    }    else
    {        return null;
    }
    Debug.Assert(list != null);    return list;
}/// <summary>/// Adds an object to the <see cref="ConcurrentBag{T}"/>./// </summary>/// <param name="item">The object to be added to the/// <see cref="ConcurrentBag{T}"/>. The value can be a null reference/// (Nothing in Visual Basic) for reference types.</param>public void Add(T item) {/ / get the local list of the thread. If the thread does not exist, create a new list (call add for the first time)
    ThreadLocalList list = GetThreadList(true);    // The actual data addition operation is performed in AddInternal
    AddInternal(list, item);
}/// <summary>/// </summary>/// <param name="list"></param>/// <param name="item"></param>private void AddInternal(ThreadLocalList list, T item){    bool lockTaken = false;    try
    {        #pragma warning disable 0420
        Interlocked.Exchange(ref list.m_currentOp, (int)ListOperation.Add);        #pragma warning restore 0420
        // Synchronization case:
        // If the list count is less than two, because it is a two-way linked list, you must obtain the lock in order to avoid conflict with any stealing thread
        // If m_needSync is set, this means that a thread needs to freeze the package and must also obtain the lock
        if (list.Count < 2 || m_needSync)
        {            // Reset it to None to avoid deadlock with the stealing thread
            list.m_currentOp = (int)ListOperation.None;            // Lock current object
            Monitor.Enter(list, ref lockTaken);
        }        // Call the ThreadLocalList.Add method to add data to the bidirectional linked list
        // If it is locked, it means that thread safety can update the Count
        list.Add(item, lockTaken);
    }    finally
    {
        list.m_currentOp = (int)ListOperation.None;        if (lockTaken)
        {
            Monitor.Exit(list);
        }
    }
}

From the above code, we can clearly know how the Add() method runs. The key is the GetThreadList() method, which can obtain the data storage list object of the current thread. If there is no data storage list, it will be automatically created or through GetUnownedList() Method to find those threads that are stopped but still store a data list, and then return the data list to the current thread to prevent memory leakage.

In the process of data addition, the fine-grained lock synchronization lock is realized, so the performance will be very high. Deletion and other operations are similar to new ones, which will not be repeated in this paper.

4. How does concurrentbag implement iterator mode

After reading the above code, I am curious about how concurrentbag < T > implements ienumeror to realize iterative access. Because concurrentbag < T > stores data through ThreadLocalList scattered in different threads, the process of implementing iterator mode will be more complex.

After reviewing the source code later, it is found that concurrentbag < T > in order to realize the iterator mode, all the data divided in different threads are saved into a list < T > collection, and then the iterator of the copy is returned. Therefore, every time the iterator is accessed, it will create a new copy of list < T >. Although it wastes a certain amount of storage space, it is logically simpler.

///< summary > / / / the local helper method releases all local list locks / / < / summary > private void releasealllocks() {/ / this method is used to release all local locks after thread synchronization
    // Release the lock occupied by traversing the ThreadLocalList object stored in each thread
    ThreadLocalList currentList = m_headList;    while (currentList != null)
    {        if (currentList.m_lockTaken)
        {
            currentList.m_lockTaken = false;
            Monitor.Exit(currentList);
        }
        currentList = currentList.m_nextList;
    }
}///< summary > / / / unfreeze the local helper method of the package from the frozen state / / < / summary > / / < param name = "locktaken" > the lock taken result from the freeze method < / parameter > private void unfreezebag (bool locktaken) {/ / first release the lock of the local variable in each thread
    // Then release the global lock
    ReleaseAllLocks();
    m_needSync = false;    if (lockTaken)
    {
        Monitor.Exit(GlobalListsLock);
    }
}///< summary > / / / the local helper function waits for all unsynchronized operations / / < / summary > private void waitalloperations(){
    Contract.Assert(Monitor.IsEntered(GlobalListsLock));

    ThreadLocalList currentList = m_headList;    // Spin and wait for other operations to complete
    while (currentList != null)
    {        if (currentList.m_currentOp != (int)ListOperation.None)
        {
            SpinWait spinner = new SpinWait();            // When there are other threads operating, cuurentOp will be set as the enumeration being operated
            while (currentList.m_currentOp != (int)ListOperation.None)
            {
                spinner.SpinOnce();
            }
        }
        currentList = currentList.m_nextList;
    }
}///< summary > / / / the local helper method obtains all local list locks / / < / summary > private void acquireallocks(){
    Contract.Assert(Monitor.IsEntered(GlobalListsLock));    bool lockTaken = false;
    ThreadLocalList currentList = m_headList;    
    // Traverse the ThreadLocalList of each thread, and then obtain the lock of the corresponding ThreadLocalList
    while (currentList != null)
    {        // Try / finally bllock to avoid threading between acquiring the lock and setting the flag taken
        try
        {
            Monitor.Enter(currentList, ref lockTaken);
        }        finally
        {            if (lockTaken)
            {
                currentList.m_lockTaken = true;
                lockTaken = false;
            }
        }
        currentList = currentList.m_nextList;
    }
}/// <summary>/// Local helper method to freeze all bag operations, it/// 1- Acquire the global lock to prevent any other thread to freeze the bag, and also new new thread can be added/// to the dictionary/// 2- Then Acquire all local lists locks to prevent steal and synchronized operations/// 3- Wait for all un-synchronized operations to be done/// </summary>/// <param name="lockTaken">Retrieve the lock taken result for the global lock, to be passed to Unfreeze method</param>private void FreezeBag(ref bool lockTaken){
    Contract.Assert(!Monitor.IsEntered(GlobalListsLock));    // Global locking safely prevents multithreaded call counting and corruption m_needSync
    Monitor.Enter(GlobalListsLock, ref lockTaken);    // This will force synchronization of any future add / perform operations
    m_needSync = true;    // Get locks for all lists
    AcquireAllLocks();    // Wait for all operations to complete
    WaitAllOperations();
}///< summary > / / / the local helper function returns the package items in the list, which is mainly used by CopyTo and ToArray. / / / this is not thread safe and should be called freezing / thawing bag blocks / / this method is private and is safe only after using Freeze/UnFreeze / / < / summary > / / < returns > list the contains the bag items < / returns > private list < T > tolist(){
    Contract.Assert(Monitor.IsEntered(GlobalListsLock));    // Create a new List
    List<T> list = new List<T>();
    ThreadLocalList currentList = m_headList;    // Traverse the ThreadLocalList in each thread and add the data of the nodes in it to the list
    while (currentList != null)
    {
        Node currentNode = currentList.m_head;        while (currentNode != null)
        {
            list.Add(currentNode.m_value);
            currentNode = currentNode.m_next;
        }
        currentList = currentList.m_nextList;
    }    return list;
}/// <summary>/// Returns an enumerator that iterates through the <see
/// cref="ConcurrentBag{T}"/>./// </summary>/// <returns>An enumerator for the contents of the <see
/// cref="ConcurrentBag{T}"/>.</returns>/// <remarks>/// The enumeration represents a moment-in-time snapshot of the contents/// of the bag.  It does not reflect any updates to the collection after /// <see cref="GetEnumerator"/> was called.  The enumerator is safe to use/// concurrently with reads from and writes to the bag./// </remarks>public IEnumerator<T> GetEnumerator(){    // Short path if the bag is empty
    if (m_headList == null)        return new List<T>().GetEnumerator(); // empty list

    bool lockTaken = false;    try
    {        // First, freeze the entire ConcurrentBag collection
        FreezeBag(ref lockTaken);        // Then ToList gets the IEnumerator of List
        return ToList().GetEnumerator();
    }    finally
    {
        UnfreezeBag(lockTaken);
    }
}

As can be seen from the above code, in order to obtain the iterator object, a total of three main operations are carried out.

use FreezeBag()Method to freeze the entire ConcurrentBag<T>Collection. Because you need to generate a collection List<T>Replica. No other thread can change the corrupted data during replica generation.

take ConcurrrentBag<T>generate List<T>Copy. Because ConcurrentBag<T>The way of storing data is special, and it is difficult to directly implement the iterator mode. Considering thread safety and logic, the best way is to generate a copy.

After completing the above operations, you can use UnfreezeBag()Method to unfreeze the entire collection.

So how does the FreezeBag() method freeze the entire collection? It is also divided into three steps.

First, obtain the global lock through Monitor.Enter(GlobalListsLock, ref lockTaken);Such a statement, so that other threads cannot freeze the collection.

Then get all threads ThreadLocalList Lock, through`AcquireAllLocks()Method so that other threads cannot operate on it and damage data.

Wait for the end of the thread that has entered the operation process WaitAllOperations()Method, which traverses each ThreadLocalList Object m_currentOp Property to ensure that all are in None Operation.

After the above process is completed, the entire concurrentbag < T > collection is frozen. To unfreeze, it is similar. I won't repeat it here.

4, Summary

The following figure describes how concurrentbag < T > stores data. Thread local storage is realized through ThreadLocal in each thread. Each thread has such a structure and does not interfere with each other. Then, m_headList in each thread always points to the first list of concurrentbag < T > and m_tailList points to the last list. M_local is used between lists The m_nextList under s is connected to form a single linked list.

The data is stored in the m_locales of each thread, and a two-way linked list is formed through the Node class. PS: note that m_tailList and m_headList are not stored in ThreadLocal, but shared by all threads.