Link to the general outline of this series of articles: surface flinger series of articles of Android GUI system
Summary & description of key points in this chapter:
The mind map of this chapter is as above. This paper mainly introduces the concept of FrameBuffer, mainly focuses on the basic concept, implementation principle and PageFlipping / double buffering technology of FrameBuffer.
1 FrameBuffer description
The Chinese name of FrameBuffer is frame buffer, which actually includes two different aspects:
- Frame: frame refers to an image. The image you see on the screen is one frame.
- Buffer: buffer is a storage area that stores frames.
The concept of FrameBuffer is very clear. It is a buffer to store the frame data of graphics / images. FrameBuffer (FB for short) is a virtual device in Linux system. The device file corresponds to / dev/fb%d (for example, / dev/fb0). This virtual device unifies the real devices implemented by different hardware manufacturers in one framework, so that the application layer can input and output graphics / images through standard interfaces. The FBD schematic is shown below:
As can be seen from the above figure, the application layer can operate the display device through standard system calls such as ioctl or mmap, which is very convenient to use. Buffer in FrameBuffer is to map the display memory in the device to the user space through mmap. Writing data on this buffer is equivalent to drawing on the screen.
Note: the framework mentioned above will lead to another concept, Linux FrameBuffer (LFB for short). LFB is a mechanism that can directly operate FB provided by Linux platform. Relying on this mechanism, the application layer can operate the display device through standard system call.
The following is the demo to realize the screenshot function in DDMS tool (directly retrieve framebuffer_service.c in the framework code), as shown below:
struct fbinfo {//Define a structure
unsigned int version;
unsigned int bpp;
unsigned int size;
unsigned int width;
unsigned int height;
unsigned int red_offset;
unsigned int red_length;
unsigned int blue_offset;
unsigned int blue_length;
unsigned int green_offset;
unsigned int green_length;
unsigned int alpha_offset;
unsigned int alpha_length;
} __attribute__((packed));
//fd is a file descriptor. The purpose of this function is to write the contents of the current screen to a file
void framebuffer_service(int fd, void *cookie)
{
struct fbinfo fbinfo;
unsigned int i, bsize;
char buf[640];
int fd_screencap;
int w, h, f;
int fds[2];
//Pipeline communication mechanism
if (pipe2(fds, O_CLOEXEC) < 0) goto pipefail;
pid_t pid = fork();
if (pid < 0) goto done;
if (pid == 0) {//Child process
dup2(fds[1], STDOUT_FILENO);
close(fds[0]);
close(fds[1]);
const char* command = "screencap";
const char *args[2] = {command, NULL};
execvp(command, (char**)args);
exit(1);
}
//PID > 0, parent process
fd_screencap = fds[0];
/* read w, h & format */
if(readx(fd_screencap, &w, 4)) goto done;
if(readx(fd_screencap, &h, 4)) goto done;
if(readx(fd_screencap, &f, 4)) goto done;
//Fill in the fbinfo structure according to the properties of the screen, which will be written to the header of the output file at last
fbinfo.version = DDMS_RAWIMAGE_VERSION;
/* see hardware/hardware.h */
/*
The following variables are related to the color format. Take RGB565 as an example to give a brief introduction.
RGB565 It means that R component is 5 bits, G component is 6 bits, B component is 5 bits, and there is no Alpha component.
The size of such a pixel is 16 bits, accounting for two bytes, which is one byte less than a pixel in RGB888 format (one pixel is three bytes).
x_length For e x ample, the R component in RGB565 is 5 bits.
x_offset The value of represents the location of the x component in memory. If one pixel accounts for two bytes in RGB565, then x \
Represents the starting position of x component in the two byte memory area, but the order is reversed, that is, B component is in front,
R In the end. So the value of red offset is 11, the value of blue offset is 0, and the value of green offset is 6.
This information is useful for format conversion (e.g. from RGB565 to RGB888).
*/
switch (f) {
case 1: /* RGBA_8888 */
fbinfo.bpp = 32;
fbinfo.size = w * h * 4;
fbinfo.width = w;
fbinfo.height = h;
fbinfo.red_offset = 0;
fbinfo.red_length = 8;
fbinfo.green_offset = 8;
fbinfo.green_length = 8;
fbinfo.blue_offset = 16;
fbinfo.blue_length = 8;
fbinfo.alpha_offset = 24;
fbinfo.alpha_length = 8;
break;
case 2: /* RGBX_8888 */
fbinfo.bpp = 32;
fbinfo.size = w * h * 4;
fbinfo.width = w;
fbinfo.height = h;
fbinfo.red_offset = 0;
fbinfo.red_length = 8;
fbinfo.green_offset = 8;
fbinfo.green_length = 8;
fbinfo.blue_offset = 16;
fbinfo.blue_length = 8;
fbinfo.alpha_offset = 24;
fbinfo.alpha_length = 0;
break;
case 3: /* RGB_888 */
fbinfo.bpp = 24;
fbinfo.size = w * h * 3;
fbinfo.width = w;
fbinfo.height = h;
fbinfo.red_offset = 0;
fbinfo.red_length = 8;
fbinfo.green_offset = 8;
fbinfo.green_length = 8;
fbinfo.blue_offset = 16;
fbinfo.blue_length = 8;
fbinfo.alpha_offset = 24;
fbinfo.alpha_length = 0;
break;
case 4: /* RGB_565 */
fbinfo.bpp = 16;
fbinfo.size = w * h * 2;
fbinfo.width = w;
fbinfo.height = h;
fbinfo.red_offset = 11;
fbinfo.red_length = 5;
fbinfo.green_offset = 5;
fbinfo.green_length = 6;
fbinfo.blue_offset = 0;
fbinfo.blue_length = 5;
fbinfo.alpha_offset = 0;
fbinfo.alpha_length = 0;
break;
case 5: /* BGRA_8888 */
fbinfo.bpp = 32;
fbinfo.size = w * h * 4;
fbinfo.width = w;
fbinfo.height = h;
fbinfo.red_offset = 16;
fbinfo.red_length = 8;
fbinfo.green_offset = 8;
fbinfo.green_length = 8;
fbinfo.blue_offset = 0;
fbinfo.blue_length = 8;
fbinfo.alpha_offset = 24;
fbinfo.alpha_length = 8;
break;
default:
goto done;
}
//Write fb information to the file header
if(writex(fd, &fbinfo, sizeof(fbinfo))) goto done;
//Write data to file
for(i = 0; i < fbinfo.size; i += bsize) {
bsize = sizeof(buf);
if (i + bsize > fbinfo.size)
bsize = fbinfo.size - i;
if(readx(fd_screencap, buf, bsize)) goto done;//Read data from FBD
if(writex(fd, buf, bsize)) goto done;//Write data to file
}
done:
TEMP_FAILURE_RETRY(waitpid(pid, NULL, 0));
close(fds[0]);
close(fds[1]);
pipefail:
close(fd);
}The purpose of this function is to take a screenshot. This example can deepen our intuitive experience of FB. Note: according to this code, we can write a simple Native executable program, and then push the ADB to the device to run.
2 Introduction to pageflipping / double buffering technology
PageFlipping mechanism will be involved in the implementation principle of FrameBuffer, so here is a simple explanation. The graph / image data is one frame with boundary. So in the production / consumption process of graphics / image data, people use a technology called PageFlipping. The Chinese name is picture exchange ("double buffer" technology), and the operation process is as follows:
- Allocate a buffer that can hold two frames of data. The first buffer is called FrontBuffer, and the second buffer is called BackBuffer.
- The consumer uses the old data in the FrontBuffer, while the producer fills the BackBuffer with the new data, and the two do not interfere with each other.
- When the display needs to be updated, the BackBuffer changes to FrontBuffer, and the FrontBuffer changes to BackBuffer.
- In this way, the latest content can always be displayed.
This process is very similar to our normal flipping action, so it is vividly called PageFlipping. PageFlipping actually uses a frame buffer queue with only two members. Later, when analyzing data transmission, we will see operations such as dequeue and queue.
3. Implementation principle of FrameBuffer
In the analysis of HWComposer, loadFbHalModule is a hardware module loaded with the management framework. Its code is as follows:
#Define Gralloc? Hardware? Module? ID "Gralloc" / / the corresponding ID loads the Gralloc module
...
int HWComposer::loadFbHalModule()
{
hw_module_t const* module;
int err = hw_get_module(GRALLOC_HARDWARE_MODULE_ID, &module);
if (err != 0) {
ALOGE("%s module not found", GRALLOC_HARDWARE_MODULE_ID);
return err;
}
return framebuffer_open(module, &mFbDev);
}Continue to analyze the implementation of framebuffer open. The code is as follows:
#define GRALLOC_HARDWARE_FB0 "fb0"
static inline int framebuffer_open(const struct hw_module_t* module,struct framebuffer_device_t** device) {
//Call the open function of the Gralloc module
return module->methods->open(module, GRALLOC_HARDWARE_FB0, (struct hw_device_t**)device);
}Generally speaking, the Gralloc module is provided by the hardware manufacturer in the actual device, so as to facilitate the best cooperation with the mobile hardware. (purpose of studying FB: better understanding of the principle of graphic display, rather than the implementation of specific hardware) research on the default implementation (only open): the directory of implementation is in hardware / libhardware / modules / granlloc, and the implementation of open is as follows:
int gralloc_device_open(const hw_module_t* module, const char* name, hw_device_t** device)
{
int status = -EINVAL;
if (!strcmp(name, GRALLOC_HARDWARE_GPU0)) {//Here is the grailloc hardware fb0,
//... take else branch, ignore here
} else {
status = fb_device_open(module, name, device);
}
return status;
}Continue to analyze FB device open. The code is as follows:
int fb_device_open(hw_module_t const* module, const char* name,
hw_device_t** device)
{
int status = -EINVAL;
if (!strcmp(name, GRALLOC_HARDWARE_FB0)) {
fb_context_t *dev = (fb_context_t*)malloc(sizeof(*dev));//Create a FB context T structure
memset(dev, 0, sizeof(*dev));
//Initializing the dev struct members and setting the FB function
dev->device.common.tag = HARDWARE_DEVICE_TAG;
dev->device.common.version = 0;
dev->device.common.module = const_cast<hw_module_t*>(module);
dev->device.common.close = fb_close;//Key point 2
dev->device.setSwapInterval = fb_setSwapInterval;
dev->device.post = fb_post;
dev->device.setUpdateRect = 0;
private_module_t* m = (private_module_t*)module;
status = mapFrameBuffer(m);//Open and read device information, key 1
if (status >= 0) {
//Get the device parameters read from the bottom layer and continue to fill the dev structure
int stride = m->finfo.line_length / (m->info.bits_per_pixel >> 3);
int format = (m->info.bits_per_pixel == 32)
? (m->info.red.offset ? HAL_PIXEL_FORMAT_BGRA_8888 : HAL_PIXEL_FORMAT_RGBX_8888)
: HAL_PIXEL_FORMAT_RGB_565;
const_cast<uint32_t&>(dev->device.flags) = 0;
const_cast<uint32_t&>(dev->device.width) = m->info.xres;
const_cast<uint32_t&>(dev->device.height) = m->info.yres;
const_cast<int&>(dev->device.stride) = stride;
const_cast<int&>(dev->device.format) = format;
const_cast<float&>(dev->device.xdpi) = m->xdpi;
const_cast<float&>(dev->device.ydpi) = m->ydpi;
const_cast<float&>(dev->device.fps) = m->fps;
const_cast<int&>(dev->device.minSwapInterval) = 1;
const_cast<int&>(dev->device.maxSwapInterval) = 1;
*device = &dev->device.common;
}
}
return status;
}@ 1 starts to analyze key point 1. The code of mapFrameBuffer is as follows:
static int mapFrameBuffer(struct private_module_t* module)
{
pthread_mutex_lock(&module->lock);
int err = mapFrameBufferLocked(module);
pthread_mutex_unlock(&module->lock);
return err;
}Continue to analyze mapFrameBufferLocked. The code is as follows:
int mapFrameBufferLocked(struct private_module_t* module)
{
if (module->framebuffer) {//Open, return directly
return 0;
}
char const * const device_template[] = {//Device list, open one
"/dev/graphics/fb%u",
"/dev/fb%u",
0 };
int fd = -1;
int i=0;
char name[64];
while ((fd==-1) && device_template[i]) {
snprintf(name, 64, device_template[i], 0);
fd = open(name, O_RDWR, 0);//open device
i++;
}
...//Get all kinds of information about the device, such as falls, info, xdpi, ydpi, fps
if (ioctl(fd, FBIOGET_FSCREENINFO, &finfo) == -1)
return -errno;
if (finfo.smem_len <= 0)
return -errno;
//Put the obtained device information into the module
module->flags = flags;
module->info = info;
module->finfo = finfo;
module->xdpi = xdpi;//x direction dpi
module->ydpi = ydpi;//y direction dpi
module->fps = fps;//Refresh rate
int err;
size_t fbSize = roundUpToPageSize(finfo.line_length * info.yres_virtual);//Byte alignment to ensure fbSize is a multiple of 4
module->framebuffer = new private_handle_t(dup(fd), fbSize, 0);
module->numBuffers = info.yres_virtual / info.yres;//Number of buffer s
module->bufferMask = 0;
//mmap is used to allocate memory and let the display output image. It only needs to copy the data to the corresponding memory vaddr
void* vaddr = mmap(0, fbSize, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
if (vaddr == MAP_FAILED) {
ALOGE("Error mapping the framebuffer (%s)", strerror(errno));
return -errno;
}
module->framebuffer->base = intptr_t(vaddr);//module structure framebuffer - > base points to the first address of mmap
memset(vaddr, 0, fbSize);
return 0;
}Next, if you want the screen to output images, just copy the data to the memory allocated by mmap.
@2. Continue to analyze the implementation of key point 2, FB post (set the function of FB). The code is as follows:
static int fb_post(struct framebuffer_device_t* dev, buffer_handle_t buffer)
{
if (private_handle_t::validate(buffer) < 0)
return -EINVAL;
fb_context_t* ctx = (fb_context_t*)dev;
private_handle_t const* hnd = reinterpret_cast<private_handle_t const*>(buffer);
private_module_t* m = reinterpret_cast<private_module_t*>(dev->common.module);
if (hnd->flags & private_handle_t::PRIV_FLAGS_FRAMEBUFFER) {//Judge whether to support multiple buffers through priv ﹣ flags ﹣ framebuffer
//If FB supports Flip, call ioctl
const size_t offset = hnd->base - m->framebuffer->base;
m->info.activate = FB_ACTIVATE_VBL;
m->info.yoffset = offset / m->finfo.line_length;//Calculate y offset
//Use the fbioput? Vscreaninfo parameter to set the currently displayed Buffer,
if (ioctl(m->framebuffer->fd, FBIOPUT_VSCREENINFO, &m->info) == -1) {
ALOGE("FBIOPUT_VSCREENINFO failed");
m->base.unlock(&m->base, buffer);
return -errno;
}
m->currentBuffer = buffer; //Point the display area to the new data frame in FB
} else {
//If Flip is not supported, copy data directly to FB
void* fb_vaddr;
void* buffer_vaddr;
//Lock FB
m->base.lock(&m->base, m->framebuffer, GRALLOC_USAGE_SW_WRITE_RARELY, 0, 0, m->info.xres, m->info.yres, &fb_vaddr);
//Lock Buffer
m->base.lock(&m->base, buffer, GRALLOC_USAGE_SW_READ_RARELY, 0, 0, m->info.xres, m->info.yres,&buffer_vaddr);
memcpy(fb_vaddr, buffer_vaddr, m->finfo.line_length * m->info.yres); //Copy data from Buffer to FB
//Unlock Buffer
m->base.unlock(&m->base, buffer);
//Unlocking FBr
m->base.unlock(&m->base, m->framebuffer);
}
return 0;
}Switching buffer through Flip mode does not aggravate flicker, but single buffer does. At the same time, devices supporting Flip should allocate at least 2 times the screen size when allocating memory. The formula for calculating the memory size is as follows:
- Finfo.line'length * info.yres'virtual
The size of the real screen height is info.yres, so if the device supports Flip, the return value of yres'virtual is 2 times or more than yres.
Finally, the conclusion is that post operation is a process of rendering buffer.
