At the end of the blog, the MonitoredProducer object, which is only a proxy, is really a BufferQueueProducer class, which is associated with BufferQueueCore and can manage up to 64 blocks of buffer. Surface can be understood as a canvas, so why should Surface be associated with a buffer queue? When playing animation, it takes at least 24 frames to form a more realistic animation effect. These data are decoded by the cpu, and it takes time to prepare them. For an image display device, the refresh cycle is fixed, and we must prepare the data when it needs it. Each frame of video play also needs to be played at a specified time, so the decoder prepares a batch of data in advance, which is stored in the buffer of the decoder memory. When the time arrives, the decoder copies the image of the internal buffer into Surface, but the display device does not immediately remove the data, so Surface also needs to be saved when the buffer arrives. Data.

Buffer Queue Core Buffer Queue Producer BufferQueueConsumer

The previous blog called the following function in Layer's onFirstRef function and created three objects, BufferQueueCore BufferQueueProducer BufferQueueConsumer. BufferCore is the core, connecting BufferQueueProducer and BufferQueueConsumer objects.

1. void BufferQueue::createBufferQueue(sp<IGraphicBufferProducer>* outProducer,  
2.         sp<IGraphicBufferConsumer>* outConsumer,  
3.         const sp<IGraphicBufferAlloc>& allocator) {  
4.   
5.     sp<BufferQueueCore> core(new BufferQueueCore(allocator));  
6.     sp<IGraphicBufferProducer> producer(new BufferQueueProducer(core));  
7.     sp<IGraphicBufferConsumer> consumer(new BufferQueueConsumer(core));  
8.   
9.     *outProducer = producer;  
10.     *outConsumer = consumer;  
11. }  

1.1 Linkages between Producers and core

The general interface of IGraphic Buffer Producer is as follows. The BufferQueueProducer class is the implementation of the interface IGraphic Buffer Producer. Before using BufferQueueProducer, connect function is called, and disconnect is called after using BufferQueueProducer.

1. class IGraphicBufferProducer : public IInterface  
2. {  
3. public:  
4.     virtual status_t requestBuffer(int slot, sp<GraphicBuffer>* buf) = 0;  
5.   
6.     virtual status_t setBufferCount(int bufferCount) = 0;  
7.   
8.     virtual status_t dequeueBuffer(int* slot, sp<Fence>* fence, bool async,  
9.             uint32_t w, uint32_t h, PixelFormat format, uint32_t usage) = 0;  
10.   
11.     virtual status_t detachBuffer(int slot) = 0;  
12.   
13.     virtual status_t detachNextBuffer(sp<GraphicBuffer>* outBuffer,  
14.             sp<Fence>* outFence) = 0;  
15.   
16.     virtual status_t attachBuffer(int* outSlot,  
17.             const sp<GraphicBuffer>& buffer) = 0;  
18.   
19.     virtual status_t queueBuffer(int slot,  
20.             const QueueBufferInput& input, QueueBufferOutput* output) = 0;  
21.     virtual void cancelBuffer(int slot, const sp<Fence>& fence) = 0;  
22.     virtual int query(int what, int* value) = 0;  
23.     virtual status_t connect(const sp<IProducerListener>& listener,  
24.             int api, bool producerControlledByApp, QueueBufferOutput* output) = 0;  
25.   
26.     virtual status_t disconnect(int api) = 0;  
27.     virtual status_t setSidebandStream(const sp<NativeHandle>& stream) = 0;  
28.     virtual void allocateBuffers(bool async, uint32_t width, uint32_t height,  
29.             PixelFormat format, uint32_t usage) = 0;  
30.     virtual status_t allowAllocation(bool allow) = 0;  
31.   
32.     virtual status_t setGenerationNumber(uint32_t generationNumber) = 0;  
33.   
34.     virtual String8 getConsumerName() const = 0;  
35. };  

A 64-item data mSlots is defined in the BufferQueueCore class.

1. BufferQueueDefs::SlotsType mSlots;  

1. typedef BufferSlot SlotsType[NUM_BUFFER_SLOTS];  

The type of each buffer is the BufferSlot type. It has two important member variables, mGraphicBuffer is a pointer to the image buffer GraphicBuffer, and mBufferState represents the state of the image buffer.

1. sp<GraphicBuffer> mGraphicBuffer;  
2. ......  
3. BufferState mBufferState;  

BufferState has the following states

1. enum BufferState {  
2.     FREE = 0,//free  
3.     DEQUEUED = 1,//The state of production, owned by the producer  
4.     QUEUED = 2,//Save data status, owned by BufferQueue  
5.     ACQUIRED = 3//Consumption status, owned by consumers  
6. };  

The dequeueBuffer function of BufferQueueProducer is used to request an idle slot from BufferQueueCore, which may or may not have a buffer. Without a buffer, the dequeueBuffer function allocates a new buffer. After you get the free slot, you also need to call the requestBuffer function to get a buffer. Get the buffer, and if you don't need it, use the cancelBuffer function to release the slot. After calling the dequeueBuffer function, the owner of the buffer is the producer, and the buffer is in the DEQUEUED state. Once the buffer replication data is completed, the control of the buffer is handed back to BufferQueueCore through the queueBuffer function, at which time the buffer will be in the QUEUED state.

1.2 Connections between Consumers and core

The following are the main functions of the IGraphic Buffer Comsumer interface:

1. virtual status_t acquireBuffer(BufferItem* outBuffer,  
2.         nsecs_t expectedPresent, uint64_t maxFrameNumber = 0) override;  
3. ......  
4.   
5. virtual status_t releaseBuffer(int slot, uint64_t frameNumber,  
6.         const sp<Fence>& releaseFence, EGLDisplay display,  
7.         EGLSyncKHR fence);  
8.   
9.   
10. virtual status_t connect(const sp<IConsumerListener>& consumerListener,  
11.         bool controlledByApp);  
12. virtual status_t disconnect();  

The BufferQueueConsumer class is an implementation of the interface IGraphicBufferComsumer. Before using it, the connect function is called to establish the connection. The parameter passed here is the IConsumerListener object, which is a callback interface. If the data in BufferQueue is ready, the onFrameAvailable function of BufferQueue is called to notify the consumer to take the data.

When taking away the data, we need to call the acquireBuffer function to change the buffer state into ACQUIRED. After using it, we can call the release Buffer function to return the buffer data to BufferQueueCore, so that the buffer becomes FREE.

1.3 Three Links

Objects BufferQueueProducer and BufferQueueConsumer do not seem to be directly related, but they are all connected together through a common BufferQueueCore object. Many operations are performed directly using BufferQueueCore object member variables rather than functions.

2. Creation of GraphicBuffer Object

For Surface, image buffer is an important data structure. It is the link between user process and image display. Let's see how Surface's image buffer is created.

2.1 Memory Buffer Creation

The dequeueBuffer function was introduced earlier. Graphic Buffer is created in this function. When a slot in space is acquired from BufferQueueCore, if there is no buffer in this slot, a new slot will be created.

Here is part of the code for the dequeueBuffer function. When you get a slot from BufferQueueCore, if you need to reallocate the image buffer, you will call the mCore - > mAllocator - > createGraphicBuffer function to recreate an image buffer.

1.     ......  
2.     *outSlot = found;//Foundcopy to outslot  
3.     ATRACE_BUFFER_INDEX(found);  
4.   
5.     attachedByConsumer = mSlots[found].mAttachedByConsumer;  
6.   
7.     mSlots[found].mBufferState = BufferSlot::DEQUEUED;//Modification of slot status to production status  
8.   
9.     const sp<GraphicBuffer>& buffer(mSlots[found].mGraphicBuffer);  
10.     if ((buffer == NULL) ||//Empty or need to be reallocated  
11.             buffer->needsReallocation(width, height, format, usage))  
12.     {  
13.         mSlots[found].mAcquireCalled = false;  
14.         mSlots[found].mGraphicBuffer = NULL;  
15.         mSlots[found].mRequestBufferCalled = false;  
16.         mSlots[found].mEglDisplay = EGL_NO_DISPLAY;  
17.         mSlots[found].mEglFence = EGL_NO_SYNC_KHR;  
18.         mSlots[found].mFence = Fence::NO_FENCE;  
19.         mCore->mBufferAge = 0;  
20.   
21.         returnFlags |= BUFFER_NEEDS_REALLOCATION;//Allocation buffer needs to be restarted  
22.     } else {  
23.         // We add 1 because that will be the frame number when this buffer  
24.         // is queued  
25.         mCore->mBufferAge =  
26.                 mCore->mFrameCounter + 1 - mSlots[found].mFrameNumber;  
27.     }  
28.   
29.     BQ_LOGV("dequeueBuffer: setting buffer age to %" PRIu64,  
30.             mCore->mBufferAge);  
31.   
32.     if (CC_UNLIKELY(mSlots[found].mFence == NULL)) {  
33.         BQ_LOGE("dequeueBuffer: about to return a NULL fence - "  
34.                 "slot=%d w=%d h=%d format=%u",  
35.                 found, buffer->width, buffer->height, buffer->format);  
36.     }  
37.   
38.     eglDisplay = mSlots[found].mEglDisplay;  
39.     eglFence = mSlots[found].mEglFence;  
40.     *outFence = mSlots[found].mFence;  
41.     mSlots[found].mEglFence = EGL_NO_SYNC_KHR;  
42.     mSlots[found].mFence = Fence::NO_FENCE;  
43.   
44.     mCore->validateConsistencyLocked();  
45. } // Autolock scope  
46.   
47. if (returnFlags & BUFFER_NEEDS_REALLOCATION) {//If you need to restart the allocation of image buffers  
48.     status_t error;  
49.     BQ_LOGV("dequeueBuffer: allocating a new buffer for slot %d", *outSlot);  
50.     sp<GraphicBuffer> graphicBuffer(mCore->mAllocator->createGraphicBuffer(//Create an image buffer  
51.             width, height, format, usage, &error));  
52.     if (graphicBuffer == NULL) {  
53.         BQ_LOGE("dequeueBuffer: createGraphicBuffer failed");  
54.         return error;  
55.     }  
56.   
57.     { // Autolock scope  
58.         Mutex::Autolock lock(mCore->mMutex);  
59.   
60.         if (mCore->mIsAbandoned) {  
61.             BQ_LOGE("dequeueBuffer: BufferQueue has been abandoned");  
62.             return NO_INIT;  
63.         }  
64.   
65.         graphicBuffer->setGenerationNumber(mCore->mGenerationNumber);  
66.         mSlots[*outSlot].mGraphicBuffer = graphicBuffer;  
67.     } // Autolock scope  
68. }  
69. ......  

The type of mAllocator is IGraphic BufferAlloc, which is also a Binder object. It is obtained in the constructor of BufferQueueCore. At this time, the allocator is empty. Specifically, the BufferQueue::createBufferQueue function is called from the constructor of Layer. At that time, the allocator parameter is empty. The mAllocator object is then created by calling the createGraphicBufferAlloc function through getComposerService.

Previous blogs have analyzed that getComposerService returns Binder objects to the Surface Flinger process, so it ends up in Surface Flinger's createGraphicBufferAlloc function (but it's somewhat unclear why Binder is used in a process).

1. ......  
2. if (allocator == NULL) {  
3.     sp<ISurfaceComposer> composer(ComposerService::getComposerService());  
4.     mAllocator = composer->createGraphicBufferAlloc();  
5.     if (mAllocator == NULL) {  
6.         BQ_LOGE("createGraphicBufferAlloc failed");  
7.     }  
8. }  
9. ......  

Let's look at Surface Flinger's createGraphicBufferAlloc function.

1. sp<IGraphicBufferAlloc> SurfaceFlinger::createGraphicBufferAlloc()  
2. {  
3.     sp<GraphicBufferAlloc> gba(new GraphicBufferAlloc());  
4.     return gba;  
5. }  

So the last call to the createGraphicBuffer function of mCore - > mAllocator in the dequeueBuffer function of BufferQueueProducer is the createGraphicBufferAlloc function of GraphicBufferAlloc.

1. sp<GraphicBuffer> GraphicBufferAlloc::createGraphicBuffer(uint32_t width,  
2.         uint32_t height, PixelFormat format, uint32_t usage, status_t* error) {  
3.     sp<GraphicBuffer> graphicBuffer(  
4.             new GraphicBuffer(width, height, format, usage));  
5.     status_t err = graphicBuffer->initCheck();  
6.     *error = err;  
7.     ......//error handling  
8.     return graphicBuffer;  
9. }  

Let's look at the initSize function called in the constructor of the GraphicBuffer object.

1. GraphicBuffer::GraphicBuffer(uint32_t inWidth, uint32_t inHeight,  
2.         PixelFormat inFormat, uint32_t inUsage)  
3.     : BASE(), mOwner(ownData), mBufferMapper(GraphicBufferMapper::get()),  
4.       mInitCheck(NO_ERROR), mId(getUniqueId())  
5. {  
6.     ......  
7.     mInitCheck = initSize(inWidth, inHeight, inFormat, inUsage);  
8. }  

In the initSize function, call alloc of GraphicBufferAllocator to allocate memory.

1. status_t GraphicBuffer::initSize(uint32_t inWidth, uint32_t inHeight,  
2.         PixelFormat inFormat, uint32_t inUsage)  
3. {  
4.     GraphicBufferAllocator& allocator = GraphicBufferAllocator::get();  
5.     uint32_t outStride = 0;  
6.     status_t err = allocator.alloc(inWidth, inHeight, inFormat, inUsage,  
7.             &handle, &outStride);  
8.     if (err == NO_ERROR) {  
9.         width = static_cast<int>(inWidth);  
10.         height = static_cast<int>(inHeight);  
11.         format = inFormat;  
12.         usage = static_cast<int>(inUsage);  
13.         stride = static_cast<int>(outStride);  
14.     }  
15.     return err;  
16. }  

Alloc calls the alloc function of the member variable mAllocDev.

1. status_t GraphicBufferAllocator::alloc(uint32_t width, uint32_t height,  
2.         PixelFormat format, uint32_t usage, buffer_handle_t* handle,  
3.         uint32_t* stride)  
4. {  
5.     ......  
6.     err = mAllocDev->alloc(mAllocDev, static_cast<int>(width),  
7.             static_cast<int>(height), format, static_cast<int>(usage), handle,  
8.             &outStride);  

The Gralloc module is loaded in the constructor of Graphic Buffer Allocator, so mAllocDev points to the Gralloc module. This will be analyzed in a later blog

1. GraphicBufferAllocator::GraphicBufferAllocator()  
2.     : mAllocDev(0)  
3. {  
4.     hw_module_t const* module;  
5.     int err = hw_get_module(GRALLOC_HARDWARE_MODULE_ID, &module);  
6.     ALOGE_IF(err, "FATAL: can't find the %s module", GRALLOC_HARDWARE_MODULE_ID);  
7.     if (err == 0) {  
8.         gralloc_open(module, &mAllocDev);  
9.     }  
10. }  

A shared memory buffer is allocated by calling alloc, and the alloc function returns the fd and pointer of the shared memory buffer. Since the buffer in Graphic Buffer is shared memory, we know that using shared memory requires passing the handle fd of shared memory. Now let's look at how to get to the customer process.

2.2 memory buffer fd passed to the client process

The GraphicBuffer class is derived from the template class Flattenable, which can be passed through Parcel. Generally, derived classes need to overload flatten and unflatten methods for object serialization and deserialization.

1. class GraphicBuffer  
2.     : public ANativeObjectBase< ANativeWindowBuffer, GraphicBuffer, RefBase >,  
3.       public Flattenable<GraphicBuffer>  

Let's first look at the flatten function. The FDS parameter is used to pass file handles. The function copies the handles into fds, so these handles can be passed to the target process through binder.

1. status_t GraphicBuffer::flatten(void*& buffer, size_t& size, int*& fds, size_t& count) const {  
2.     size_t sizeNeeded = GraphicBuffer::getFlattenedSize();  
3.     if (size < sizeNeeded) return NO_MEMORY;  
4.   
5.     size_t fdCountNeeded = GraphicBuffer::getFdCount();  
6.     if (count < fdCountNeeded) return NO_MEMORY;  
7.   
8.     int32_t* buf = static_cast<int32_t*>(buffer);  
9.     buf[0] = 'GBFR';  
10.     buf[1] = width;  
11.     buf[2] = height;  
12.     buf[3] = stride;  
13.     buf[4] = format;  
14.     buf[5] = usage;  
15.     buf[6] = static_cast<int32_t>(mId >> 32);  
16.     buf[7] = static_cast<int32_t>(mId & 0xFFFFFFFFull);  
17.     buf[8] = static_cast<int32_t>(mGenerationNumber);  
18.     buf[9] = 0;  
19.     buf[10] = 0;  
20.   
21.     if (handle) {  
22.         buf[9] = handle->numFds;  
23.         buf[10] = handle->numInts;  
24.         memcpy(fds, handle->data,//Copy the middle of handle into fds  
25.                 static_cast<size_t>(handle->numFds) * sizeof(int));  
26.         memcpy(&buf[11], handle->data + handle->numFds,  
27.                 static_cast<size_t>(handle->numInts) * sizeof(int));  
28.     }  
29.   
30.     buffer = static_cast<void*>(static_cast<uint8_t*>(buffer) + sizeNeeded);  
31.     size -= sizeNeeded;  
32.     if (handle) {  
33.         fds += handle->numFds;  
34.         count -= static_cast<size_t>(handle->numFds);  
35.     }  
36.   
37.     return NO_ERROR;  
38. }  

Look at the unflatten function. When calling this function, the file handle of the shared area is ready, but the memory has not been mapped yet. The mBufferMapper.registerBuffer function is called to map the memory.

1. status_t GraphicBuffer::unflatten(  
2.         void const*& buffer, size_t& size, int const*& fds, size_t& count) {  
3.     ......  
4.     if (handle != 0) {  
5.         status_t err = mBufferMapper.registerBuffer(handle);  
6.         ......  
7.     }  
8.   
9.     buffer = static_cast<void const*>(static_cast<uint8_t const*>(buffer) + sizeNeeded);  
10.     size -= sizeNeeded;  
11.     fds += numFds;  
12.     count -= numFds;  
13.   
14.     return NO_ERROR;  
15. }  

Let's look at the Graphic Buffer Mapper:: registerBuffer function, which calls the registerBuffer function of mAllocMod. mAllocMod also refers to the pointer to the Gralloc module, which is created in the constructor of Graphic Buffer Mapper, so it actually calls the gralloc_register_buffer function of the Gralloc module. This function calls mmap to map shared memory. We will analyze it in detail in the following blogs.

1. status_t GraphicBufferMapper::registerBuffer(buffer_handle_t handle)  
2. {  
3.     ATRACE_CALL();  
4.     status_t err;  
5.   
6.     err = mAllocMod->registerBuffer(mAllocMod, handle);  
7.   
8.     ALOGW_IF(err, "registerBuffer(%p) failed %d (%s)",  
9.             handle, err, strerror(-err));  
10.     return err;  
11. }  

In the case of hardware equipment supporting frame buffer, the buffer of drawing graphics in Surface is the buffer of frame buffer. After drawing, if no image synthesis is needed, only flip operation is needed to complete the output of the image, and no copy process is needed in the middle, which is very efficient.