[Roll] EasyPlayer Realizes Synchronized Video Recording while Playing

Keywords: github Android Mobile

In the previous blog Easy Pusher implemented Android Mobile Live push synchronization video function. http://blog.csdn.net/jyt0551/article/details/58714595 ) I wrote about Easy Pusher's ability to push and store local videos at the same time. Let's introduce Easy Player's ability to save local videos today. EasyPlayer also uses MediaMuxer for video recording. Unlike EasyPusher, Player keeps remote audio and video streams. Player currently supports the storage of video in H264 format and audio in AAC format.

In the previous blog, the metadata of audio and video stream, MediaFormat, was extracted from MediaCodec. That is to say, the hard-coded library provides the interface of metadata for acquiring audio and video. But unfortunately we don't have such a convenient excuse to call on the playback side. The MediaFormat object can only be constructed manually.

The implementation of MediaFormat class is very simple. It encapsulates the parameters of audio and video in the form of key-value pairs and provides an interface for reading and writing. So we can create a MediaFormat object and assign it with specific parameters. According to the author's research, the metadata needed for video streaming is shown in the following table.

data Explain
KEY_MIME Video MIME, such as video/avc
width width
height height
csd-0 SPS
csd-1 PPS

For audio, the following information is needed:

data Explain
KEY_MIME Audio MIME, such as audio/mp4a-latm
KEY_CHANNEL_COUNT Channel number
KEY_SAMPLE_RATE sampling rate
csd-0 Some more details, such as profile, index of sample, etc. Processing of Audio Data Referring to exoplayer
csd-1 This. More details are not clear. The author also refers to the processing in exoplayer.

With this basic information in mind, the next thing we need to do is get this information from the stream and build a MediaFormat to add Video or Audio Track.
Following is a video-related information, add a VideoTrack code.

// Add Video Track
MediaFormat format = new MediaFormat();
format.setInteger(MediaFormat.KEY_WIDTH, mWidth);
format.setInteger(MediaFormat.KEY_HEIGHT, mHeight);
mCSD0.clear();
format.setByteBuffer("csd-0", mCSD0);
mCSD1.clear();
format.setByteBuffer("csd-1", mCSD1);
format.setString(MediaFormat.KEY_MIME, "video/avc");


Log.i(TAG, String.format("addTrack video track :%s", format));
mMuxerVideoTrack = muxer.addTrack(format);

Below is the code to add AudioTrack.

int audioObjectType = 2;
byte[] audioSpecificConfig = CodecSpecificDataUtil.buildAacAudioSpecificConfig(audioObjectType, sampleRateIndex, channelConfig);
Pair<Integer, Integer> audioParams = CodecSpecificDataUtil.parseAacAudioSpecificConfig(audioSpecificConfig);
//                                format.setInteger(MediaFormat.KEY_MAX_INPUT_SIZE, 0);
format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AAC);
format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, audioParams.second);
format.setInteger(MediaFormat.KEY_SAMPLE_RATE, audioParams.first);

List<byte[]> bytes = Collections.singletonList(audioSpecificConfig);
for (int j = 0; j < bytes.size(); j++) {
    format.setByteBuffer("csd-" + j, ByteBuffer.wrap(bytes.get(j)));
}

Log.i(TAG, String.format("addTrack audio track :%s", format));
mMuxerAudioTrack = muxer.addTrack(format);

So far, the audio and video channels have been added, and the next step is to write data. The code is as follows:

private synchronized void pumpSample(RTSPClient.FrameInfo frameInfo) {
      if (mObject == null) return;
      if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR2) {
          MediaMuxer muxer = (MediaMuxer) mObject;
          MediaCodec.BufferInfo bi = new MediaCodec.BufferInfo();
          bi.offset = frameInfo.offset;
          bi.size = frameInfo.length;
          ByteBuffer buffer = ByteBuffer.wrap(frameInfo.buffer, bi.offset, bi.size);
          bi.presentationTimeUs = frameInfo.stamp;
          try {
              if (frameInfo.audio) {
                  bi.offset += 7;
                  bi.size -= 7;
                  if (mMuxerAudioTrack > -1)
                      muxer.writeSampleData(mMuxerAudioTrack, buffer, bi);
              } else if (mMuxerVideoTrack > -1) {
                  if (frameInfo.type != 1) {
                      bi.flags = 0;
                  } else {
                      bi.flags = MediaCodec.BUFFER_FLAG_KEY_FRAME;
                  }
                  muxer.writeSampleData(mMuxerVideoTrack, buffer, bi);
              }
          } catch (IllegalStateException ex) {
              ex.printStackTrace();
          } catch (IllegalArgumentException ex) {
              ex.printStackTrace();
          }
      }
  }

Finally, after the video is completed, close MediaMuxer and release related resources.

    public synchronized void stopRecord() {
        mMuxerAudioTrack = mMuxerVideoTrack = -1;
        mRecordingPath = null;
        if (mObject == null) return;
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR2) {
            MediaMuxer muxer = (MediaMuxer) mObject;
            try {
                muxer.release();
            } catch (IllegalStateException ex) {
                ex.printStackTrace();
            } catch (IllegalArgumentException ex) {
                ex.printStackTrace();
            }
        }
        mObject = null;

        ResultReceiver rr = mRR;
        if (rr != null) {
            rr.send(RESULT_RECORD_END, null);
        }
    }

Download address

EasyPlayer version download: https://fir.im/EasyPlayer
Demo source download: https://github.com/EasyDarwin/EasyPlayer_Android
Other platform source code below: https://github.com/EasyDarwin/EasyPlayer

EasyPlayerPro version download: https://fir.im/EasyPlayerPro

Get more information

Mail: support@easydarwin.org

WEB: www.EasyDarwin.org

Copyright © EasyDarwin.org 2012-2017

![EasyDarwin](http://www.easydarwin.org/skin/easydarwin/images/wx_qrcode.jpg

Posted by StirCrazy on Wed, 10 Apr 2019 13:48:31 -0700