The previous article introduced YUV format and gave an example of YUYV422 to RGB24. In fact, FFmpeg has a function dedicated to image format conversion. This article introduces how to use FFmpeg conversion, because AVFrame structure is also used in conversion, so AVFrame will also be introduced here. AVFrame is an important structure in FFmpeg.
AVFrame, as the name suggests, this structure should store the information of video frames. Like a frame image, it can also be saved in the AVFrame structure. In fact, we can read a YUV image data into AVFrame directly from a YUV file. The examples later in this article do the same.
In order to understand how AVFrame stores a YUV image (of course, AVFrame can store images in other formats), let's take a look at the main members of the AVFrame structure.
typedef struct AVFrame
{
#define AV_NUM_DATA_POINTERS 8
uint8_t * data [AV_NUM_DATA_POINTERS]; //Point to image data
int linesize [AV_NUM_DATA_POINTERS]; //Length of row
int width; //Width of image
int height; //Image height
int format; //Image format
......
}AVFrame;
Notice that the data member is an array of pointers. The content it points to is the actual data of the image.
You can use av_frame_alloc(void) function to allocate an AVFrame structure. This function only allocates the AVFrame structure, but the memory pointed to by data is not allocated, which needs to be specified. The size of this memory is the size required for a specific format image, such as Previous blog post As mentioned in, for YUYV422 format, the required size is width * height * 2. Therefore, the whole initialization process of AVFrame structure is as follows:
AVFrame* frame = av_frame_alloc(); //Here, FFmpeg will help us calculate how many bytes are needed to store the image in this format //It is equivalent to width * height * 2 in the example of the previous blog post int bytes_num = avpicture_get_size(AV_PIX_FMT_YUV420P, width, height); //AV_PIX_FMT_YUV420P is a macro definition defined by FFmpeg indicating YUV420P image format //Apply for space to store picture data. Contains source and target data uint8_t* buff = (uint8_t*)av_malloc(bytes_num); //Front AV_ frame_ The alloc function only allocates memory for the AVFrame structure, //The memory pointed to by this type of pointer has not been allocated. Here put AV_ The memory obtained by malloc is associated with AVFrame. //Of course, it also sets other members of AVFrame avpicture_fill((AVPicture*)frame, buff, AV_PIX_FMT_ YUV420P,width, height);
Seeing this, some readers may wonder: the data member is a pointer array (that is, every element in the array is a pointer), how can a buff be enough (many to one relationship). In fact, this is a cleverness of FFmpeg design. Remember Previous blog post What are the physical storage of images Planar and packed? This data pointer array is designed for planar. YUV for planar mode. data[0] points to the start position of Y component, data[1] points to the start position of U component, and data[2] points to the start position of V component. For the packed mode YUV, data[0] points to the start of the data, while data[1] and data[2] are NULL.
In the above code, run avpicture_ After fill, data[0] will point to the start of buff, that is, data[0] is equal to buff. data[1] points to a position of the buff array (the position is the beginning of the U component), and data[2] also points to a position of the buff array (the position is the beginning of the V component).
Some netizens said that for planar mode, you need to read and write separately. In fact, both planar and packed modes are using acpicture_ After the fill function is processed, the following method can be used to read the data of an image into AVFrame without reading data[0], data[1] and data[2] respectively. Because for an image file, if it is an image format in plannar mode, its storage must first save all Y of an image, then save all U of another image, and then save all V of an image. This just corresponds to the three pointers of the data array.
The same is true for writing images. There is no need to divide data[0], data[1] and data[2].fread(frame->data[0], 1, bytes_num, fin);
After talking so much, I haven't said how FFmpeg converts the image format. Now let's talk about it.
FFmpeg defines a structure SwsContext, which records the format and size of the source image and the target image during image format conversion. Then use SWS_ The scale function can be directly converted. The process is as follows:
SwsContext* sws_ctx = sws_getContext(src_width, src_height,
AV_PIX_FMT_YUV420P,
dst_width, dst_height,
AV_PIX_FMT_YUYV422,
SWS_BICUBIC,
NULL,
NULL,
NULL);
sws_scale(sws_ctx, src_frame->data, src_frame->linesize,
0, height, //Height of source image
dst_frame->data, dst_frame->linesize);A complete conversion example is given below. This example converts YUV420P to YUYV422 and writes it to a file.
#ifdef __cplusplus
#define __STDC_CONSTANT_MACROS
#ifdef _STDINT_H
#undef _STDINT_H
#endif
# include <stdint.h>
#endif
extern "C"
{
#include<libavcodec/avcodec.h>
#include<libavformat/avformat.h>
#include<libavutil/log.h>
#include<libswscale/swscale.h>
}
#include<stdio.h>
#include <windows.h> //for saveAsBitmap
bool saveAsBitmap(AVFrame *pFrameRGB, int width, int height, int iFrame)
{
FILE *pFile = NULL;
BITMAPFILEHEADER bmpheader;
BITMAPINFO bmpinfo;
char fileName[32];
int bpp = 24;
// open file
sprintf(fileName, "frame%d.bmp", iFrame);
pFile = fopen(fileName, "wb");
if (!pFile)
return false;
bmpheader.bfType = ('M' <<8)|'B';
bmpheader.bfReserved1 = 0;
bmpheader.bfReserved2 = 0;
bmpheader.bfOffBits = sizeof(BITMAPFILEHEADER) + sizeof(BITMAPINFOHEADER);
bmpheader.bfSize = bmpheader.bfOffBits + width*height*bpp/8;
bmpinfo.bmiHeader.biSize = sizeof(BITMAPINFOHEADER);
bmpinfo.bmiHeader.biWidth = width;
bmpinfo.bmiHeader.biHeight = -height; //reverse the image
bmpinfo.bmiHeader.biPlanes = 1;
bmpinfo.bmiHeader.biBitCount = bpp;
bmpinfo.bmiHeader.biCompression = BI_RGB;
bmpinfo.bmiHeader.biSizeImage = 0;
bmpinfo.bmiHeader.biXPelsPerMeter = 100;
bmpinfo.bmiHeader.biYPelsPerMeter = 100;
bmpinfo.bmiHeader.biClrUsed = 0;
bmpinfo.bmiHeader.biClrImportant = 0;
fwrite(&bmpheader, sizeof(BITMAPFILEHEADER), 1, pFile);
fwrite(&bmpinfo.bmiHeader, sizeof(BITMAPINFOHEADER), 1, pFile);
uint8_t *buffer = pFrameRGB->data[0];
for (int h=0; h<height; h++)
{
for (int w=0; w<width; w++)
{
fwrite(buffer+2, 1, 1, pFile);
fwrite(buffer+1, 1, 1, pFile);
fwrite(buffer, 1, 1, pFile);
buffer += 3;
}
}
fclose(pFile);
return true;
}
int main(int argc, char** argv)
{
const char* filename = argc > 1 ? argv[1] : "flower_cif.yuv";
FILE* fin = fopen(filename, "rb");
if( fin == NULL )
{
printf("can't open the file\n");
return -1;
}
int width = 352;
int height = 288;
AVPixelFormat src_fmt = AV_PIX_FMT_YUV420P;
AVPixelFormat dst_fmt = AV_PIX_FMT_YUYV422;
AVFrame* src_frame = av_frame_alloc();
AVFrame* dst_frame = av_frame_alloc();
if( src_frame == NULL || dst_frame == NULL )
{
printf("av_frame_alloc fail\n");
return -1;
}
//Here, FFmpeg will help us calculate how many bytes are needed to store the image in this format
//It is equivalent to width * height * 2 in the previous example
int src_bytes_num = avpicture_get_size(src_fmt,
width, height);
int dst_bytes_num = avpicture_get_size(dst_fmt,
width, height);
//Apply for space to store picture data. Contains source and target data
uint8_t* src_buff = (uint8_t*)av_malloc(src_bytes_num);
uint8_t* dst_buff = (uint8_t*)av_malloc(dst_bytes_num);
//Front AV_ frame_ The alloc function only allocates memory for the AVFrame structure,
//The memory pointed to by this type of pointer has not been allocated. Here put AV_ The memory obtained by malloc is associated with AVFrame.
//Of course, it also sets other members of AVFrame
avpicture_fill((AVPicture*)src_frame, src_buff, src_fmt,
width, height);
avpicture_fill((AVPicture*)dst_frame, dst_buff, dst_fmt,
width, height);
//This section mainly explains the meaning of the linesize member. If you don't want to see it, you can ignore it
//There is a very important equivalence relationship in YUV format, that is, there are as many y as there are pixels.
//Linesize, as its name implies, is the size of a line (that is, a line). For yuv420p. data[0] stores y, and correspondingly linesize[0] is
//Indicates how many y's are in a row. For 352 * 288 images, there are 352 pixels in a row. According to the equivalence relationship just now. Then linesize[0] is
//It should be 352. That is, there are 352 y's in a row. For linesize[1], because data[1] stores u. And a row of 352 pixels in yuv420p format,
//It only needs 352 / 2, or 176. Therefore, the size of linesize[1] is 176. Similarly, the linesize[2] is 176.
//For yuyv422 format. The data[0] line is responsible for storing y, u and V components. And y:u:v = 2:1:1. According to what I said earlier
//In the same quantity relationship, y is equal to 352 (relative to the image of 352 * 288), and u and v are equal to 352 / 2. So u+v equals 352. So linesize[0]
//Equal to 352 * 2
printf("%d %d %d\n", src_frame->linesize[0],
src_frame->linesize[1], src_frame->linesize[2]);
printf("%d %d %d \n", dst_frame->linesize[0],
dst_frame->linesize[1], dst_frame->linesize[2]);
//Configure the transformation. Here, you need to set the size and format of the conversion source and the size and format of the conversion target
//After setting, you can use SWS directly below_ Scale function to convert
SwsContext* sws_ctx = sws_getContext(width, height,
src_fmt,
width, height,
dst_fmt,
SWS_BICUBIC,
//SWS_BILINEAR,
NULL,
NULL,
NULL);
if( sws_ctx == NULL)
{
printf("sws_getContext fail ");
return -1;
}
FILE* fout = fopen("yuyv422.yuv", "wb");
int count = 0;
while( 1 )
{
int ret = fread(src_frame->data[0], 1, src_bytes_num, fin);
if( ret != src_bytes_num )
{
printf("don't read enough data %d\n", ret);
break;
}
sws_scale(sws_ctx, src_frame->data, src_frame->linesize,
0, height,
dst_frame->data, dst_frame->linesize);
ret = fwrite(dst_frame->data[0], 1, dst_bytes_num, fout);
if( ret != dst_bytes_num )
printf("don't write enough data %d \n", ret);
//If you want to save to BMP format, set the format of the target image to RGB24.
//Just put the front AVPixelFormat dst_fmt = AV_PIX_FMT_YUYV422;
//Change to AVPixelFormat dst_fmt = AV_PIX_FMT_RGB24; that will do
saveAsBitmap(dst_frame, width, height, count++);
}
av_free(src_frame);
av_free(dst_frame);
av_free(src_buff);
av_free(dst_buff);
sws_freeContext(sws_ctx);
fclose(fin);
fclose(fout);
return 0;
}
In the example, you can also save the image as a bmp image. Which function comes from: http://blog.csdn.net/ajaxhe/article/details/7340508.
The YUV420P format file used in the example can be found at here Download.
reference resources:
https://blog.csdn.net/luotuo44/article/details/26486877