abstract

The Image representation in Swift is not only Image, but also a more underlying way, such as CVPixelBuffer pixel cache. If the pixel cache is converted into CGImage that can be displayed in the application, you need to know what processing is involved.

CGImage Apple's official explanation is a bitmap Image or Image mask. It is an attribute in the UIImage class and can be called an Image object through the initialization function of UIImage.

CVPixelBuffer is the reference of the core cache pixel object, where an image is stored.

In some application scenarios, CVPixelBuffer needs to be converted into CGImage for display.

CVPixelBuffer To CGImage

CVPixelBuffer conversion obtains CGimage objects, which can be processed by vtcreatecgimagefromcvpixelbuffer (pixelbuffer:, options:, ImageOut:) - > osstatus function, but not all CVPixelBuffer objects can support conversion.

public static func create(pixelBuffer: CVPixelBuffer) -> CGImage? {
  var cgImage: CGImage?
  VTCreateCGImageFromCVPixelBuffer(pixelBuffer, options: nil, imageOut: &cgImage)
  return cgImage
}

Before using the above functions, import the framework declaration VideoToolbox. After all, it is not a function of the UIKit framework.

import VideoToolbox

In addition, you can also use Core Image as a bridge to indirectly achieve the conversion effect. The object processing of CIImage(cvPixelBuffer:) and CIContext are used here.

public static func create(pixelBuffer: CVPixelBuffer, context: CIContext) -> CGImage? {
  let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
  let rect = CGRect(x: 0, y: 0, width: CVPixelBufferGetWidth(pixelBuffer),
                                height: CVPixelBufferGetHeight(pixelBuffer))
  return context.createCGImage(ciImage, from: rect)
}

If no context object is passed in, you can directly create a CIContext object. This object is only used as a canvas for drawing, and there is no very profound place.

Using this function also requires importing the framework CoreImage.

The last way to achieve the effect of conversion is to use the CGContext method, which can create a bitmap CGContext using the CVPixelBuffer object storage. The pixel format can support 32ARGB, or change the variables of bitmapInfo and space attributes to change the pixel format. This method is more flexible and relatively complex. If you have energy, you can properly understand it.

// Remember to import the frame before using it
import CoreGraphics

public static func create(pixelBuffer: CVPixelBuffer) -> CGImage? {

  guard kCVReturnSuccess == CVPixelBufferLockBaseAddress(pixelBuffer, .readOnly) else {
    return nil
  }
  defer { CVPixelBufferUnlockBaseAddress(pixelBuffer, .readOnly) }

  if let context = CGContext(data: CVPixelBufferGetBaseAddress(pixelBuffer),
                             width: CVPixelBufferGetWidth(pixelBuffer),
                             height: CVPixelBufferGetHeight(pixelBuffer),
                             bitsPerComponent: 8,
                             bytesPerRow: CVPixelBufferGetBytesPerRow(pixelBuffer),
                             space: CGColorSpaceCreateDeviceRGB(),
                             bitmapInfo: CGImageAlphaInfo.noneSkipFirst.rawValue),
     let cgImage = context.makeImage() {
    return cgImage
  } else {
    return nil
  }
}

The properties in the initialization function of the context object created in the code are all necessary information in an image. If you are interested in this, leave me a message and I will talk about the image in detail.

Digression

In a hurry, what you said may not be comprehensive. If you encounter any problems in the process of viewing, leave me a message in the comment area and I will reply as soon as possible