Still and Video Media Capture

Most of the content comes from AV Foundation Development Secret Book.

AVFoundation capture class

Capture session

AVCaptureSession captures sessions that are equivalent to a virtual "plugboard" for connecting input and output resources. Capture session management data streams from physical devices. For example, cameras and microphone devices are exported to one or more destinations. The input and output routes can be dynamically configured to allow developers to rewrite the configuration capture environment as needed during the session.

AVCaptureSession can be configured with an additional session preset to control the format and quality of captured data. The session default value is AVCaptureSessionPresetHigh by default. Other values are:

Identification	Resolving power	Notes
AVCaptureSessionPresetHigh	High	The highest recording quality, each device is different
AVCaptureSessionPresetMedium	Medium	For wifi sharing, the actual value may change
AVCaptureSessionPresetLow	Low	For 3G sharing, the actual value may change
AVCaptureSessionPreset640x480	640x480	VGA.
AVCaptureSessionPreset1280x720	1280x720	720p HD.
AVCaptureSessionPresetPhoto	Photo	Complete photo resolution, no support for video output

Create capture sessions as follows:

    self.captureSession = [[AVCaptureSession alloc] init];
    self.captureSession.sessionPreset = AVCaptureSessionPresetHigh;

If you want to configure a media frame to a specific size, you should check whether it supports it before setting it, as follows:

if ([session canSetSessionPreset:AVCaptureSessionPreset1280x720]) {
    session.sessionPreset = AVCaptureSessionPreset1280x720;
} else {
    // Handle the failure.
}

If you want to reconfigure a session, or make changes to an existing session, then the changes should be placed between the beginConfiguration and commitConfiguration methods. For example, switching the front and rear cameras requires rewriting the configuration capture session:

        [self.captureSession beginConfiguration];

        [self.captureSession removeInput:self.activeVideoInput];

        if ([self.captureSession canAddInput:videoInput]) {
            [self.captureSession addInput:videoInput];
            self.activeVideoInput = videoInput;
        }else{
            [self.captureSession addInput:self.activeVideoInput];
        }

        [self.captureSession commitConfiguration];

Start and Stop Sessions

Start session
The startRunning method is a synchronous call that takes a certain amount of time, so it should be called in an asynchronous way in the videoQueue queue, so that the main thread will not be blocked:

- (void)startSession {

    if (![self.captureSession isRunning]) {
        dispatch_async(self.videoQueue, ^{
            [self.captureSession startRunning];
        });
    }

}

Stop session

- (void)stopSession {

    if ([self.captureSession isRunning]) {
        dispatch_async(self.videoQueue, ^{
            [self.captureSession stopRunning];
        });
    }

}

Monitor capture session state

Capturing a session sends out notifications that you can listen to, for example, when it starts or stops running or interrupts. You can register to receive AVCapture Session Runtime Error Notification to handle errors. You can also query the running property of the session to determine whether it is running, and the interrupted property to determine whether it is interrupted. In addition, the attributes of running and interrupted both conform to KVC, and notifications are published on the main thread.

Capture equipment

AVCapture Device defines an interface for physical devices such as cameras or microphones, and defines a large number of control methods for physical hardware devices, such as controlling camera focus, exposure, white balance and flash.

AVCaptureDevice defines a number of methods for accessing capture devices of the system, such as AVCaptureDevice class methods devices and devicesWithMediaType: Find out the currently available capture devices.

Most commonly used is defaultDeviceWithMediaType:, which returns a system-specified default device based on a given media type, such as:

AVCaptureDevice *videoDevice = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];

The above code requests a default video device, which returns to the rear camera under the iOS system with the front and rear cameras.

Device characteristics

You can query the different features of the device. You can also test whether it provides a specific media type or supports the presupposition of a given capture session, using methods hasMediaType: and supportsAVCaptureSessionPreset:

The following code shows how to get all the available devices and output their names. If it's a video device, output their locations.

NSArray *devices = [AVCaptureDevice devices];

for (AVCaptureDevice *device in devices) {
    NSLog(@"Device name: %@", [device localizedName]);
    if ([device hasMediaType:AVMediaTypeVideo]) {
        if ([device position] == AVCaptureDevicePositionBack) {
            NSLog(@"Device position : back");
        } else {
            NSLog(@"Device position : front");
        }
    }
}

Capture device input

Before processing with capture devices, you first need to add it as input to capture sessions. However, a capture device cannot be added directly to the AVCaptureSession, and can be added by encapsulating it in an AVCaptureDeviceInput instance. This object acts as a wiring board between device output data and capture sessions.
Create AVCaptureDeviceInput using deviceInputWithDevice: error: method

AVCaptureDeviceInput *videoInput = [AVCaptureDeviceInput deviceInputWithDevice:videoDevice error:error];

A valid NSError pointer needs to be passed to this method, because the error description information encountered in input creation will be reflected here.

Capture device input added to AVCaptureSession

    AVCaptureDevice *videoDevice = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo];
    AVCaptureDeviceInput *videoInput = [AVCaptureDeviceInput deviceInputWithDevice:videoDevice error:error];
    //Add to the session
    if (videoDevice) {
        if ([self.captureSession canAddInput:videoInput]) {
            [self.captureSession addInput:videoInput];
            self.activeVideoInput = videoInput;
        }
    }

Capture device output

To capture the output of the capture device, one or more outputs need to be added. AVCaptureOutput is an abstract base class for finding output destinations for data captured from a session. The framework defines some advanced extension classes of this abstract class:

• AVCapture Still Image Output Captures Static Pictures
• AVCaptureMovieFileOutput Captures Video
• AVCapture Audio Data Output if you want to process the audio data being captured
• AVCaptureVideoData Output If you want to process frames in captured videos, for example, create your own custom view layer

Add output to capture session

    //Set static image output to capture jpeg format images
    self.imageOutput = [[AVCaptureStillImageOutput alloc] init];
    self.imageOutput.outputSettings = @{AVVideoCodecKey : AVVideoCodecJPEG};
    if([self.captureSession canAddOutput:self.imageOutput]){
        [self.captureSession addOutput:self.imageOutput];
    }

    //Setting movie file output
    self.movieOutput = [[AVCaptureMovieFileOutput alloc] init];
    if ([self.captureSession canAddOutput:self.movieOutput]) {
        [self.captureSession addOutput:self.movieOutput];
    }

Capture connection

AVCaptureConnection represents the link between input and output

Access to connections allows developers to control the underlying flow of signals, such as disabling certain links or accessing individual audio tracks in audio connections.

Capture Preview

AVCaptureVideoPreview Layer is a subclass of CALayer, which is used to preview captured video data in real time.

Initialize AVCaptureVideoPreview Layer in the following way

AVCaptureVideoPreviewLayer *captureVideoPreviewLayer = [[AVCaptureVideoPreviewLayer alloc] initWithSession:captureSession];

Resources

• AV Foundation Development Secret Books
• Still and Video Media Capture
• [AVFoundation] Image Capture and Video Recording