Originally, I didn't want to write the initialization process. I think there are many articles on the Internet to analyze. But in the whole process of analysis, it is revealed that some problems are not well understood, so it is necessary to do it once.
The first is the java layer:
private void initPlayer(IjkLibLoader libLoader) { loadLibrariesOnce(libLoader); initNativeOnce(); Looper looper; if ((looper = Looper.myLooper()) != null) { mEventHandler = new EventHandler(this, looper); } else if ((looper = Looper.getMainLooper()) != null) { mEventHandler = new EventHandler(this, looper); } else { mEventHandler = null; } /* * Native setup requires a weak reference to our object. It's easier to * create it here than in C++. */ native_setup(new WeakReference<IjkMediaPlayer>(this)); }
There are actually two things, one is load Libraries Once, the other is init NativeOnce. The former code is not pasted, that is, load Library 3 so, ijkffmpeg, ijksdl and ijkplayer. ffmpeg protocol and encoding and decoding, sdl tube rendering and display, ijkplayer management player. Every call to load Library goes to each so's JNI_OnLoad function, which means that the initial initialization of the three so's is handled in the function JNI_OnLoad. Look back; in the latter case, the init NativeOne actually goes native_init. This corresponds to jni's function IjkMediaPlayer_native_init. In ijkplayer_jni.c:
static void IjkMediaPlayer_native_init(JNIEnv *env) { MPTRACE("%s\n", __func__); }
Nothing has been done, has it?
Come back and see what JNI_OnLoad has done:
JNIEXPORT jint JNI_OnLoad(JavaVM *vm, void *reserved) { JNIEnv* env = NULL; g_jvm = vm; if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) { return -1; } assert(env != NULL); pthread_mutex_init(&g_clazz.mutex, NULL ); // FindClass returns LocalReference IJK_FIND_JAVA_CLASS(env, g_clazz.clazz, JNI_CLASS_IJKPLAYER); (*env)->RegisterNatives(env, g_clazz.clazz, g_methods, NELEM(g_methods) ); ijkmp_global_init(); ijkmp_global_set_inject_callback(inject_callback); FFmpegApi_global_init(env); return JNI_VERSION_1_4; }
The preceding are some common practices, mainly registering function tables, which are used to call c-layer functions in the java layer. Then it's ijkmp_global_init, which eventually goes to ffp_global_init:
void ffp_global_init() { if (g_ffmpeg_global_inited) return; /* register all codecs, demux and protocols */ avcodec_register_all(); #if CONFIG_AVDEVICE avdevice_register_all(); #endif #if CONFIG_AVFILTER avfilter_register_all(); #endif av_register_all(); ijkav_register_all(); avformat_network_init(); av_lockmgr_register(lockmgr); av_log_set_callback(ffp_log_callback_brief); av_init_packet(&flush_pkt); flush_pkt.data = (uint8_t *)&flush_pkt; g_ffmpeg_global_inited = true; }
Basically, ffmpeg's initial content is the most, and av should be the beginning. Register the decoder and then register the protocol. Let's look at ijkav_register_all:
void ijkav_register_all(void) { static int initialized; if (initialized) return; initialized = 1; av_register_all(); /* protocols */ av_log(NULL, AV_LOG_INFO, "===== custom modules begin =====\n"); #ifdef __ANDROID__ IJK_REGISTER_PROTOCOL(ijkmediadatasource); #endif IJK_REGISTER_PROTOCOL(async); IJK_REGISTER_PROTOCOL(ijklongurl); IJK_REGISTER_PROTOCOL(ijktcphook); IJK_REGISTER_PROTOCOL(ijkhttphook); IJK_REGISTER_PROTOCOL(ijksegment); /* demuxers */ IJK_REGISTER_DEMUXER(ijklivehook); av_log(NULL, AV_LOG_INFO, "===== custom modules end =====\n"); }
Basically, it is for protocol registration to support network transmission. Then avformat_network_init, ffmpeg's network initialization. Finally, it reaches ff_network_init, which is a WSAStartup.
Look back at the registration of so many protocols, first look at this macro:
#define IJK_REGISTER_PROTOCOL(x) \ { \ extern URLProtocol ijkimp_ff_##x##_protocol; \ int ijkav_register_##x##_protocol(URLProtocol *protocol, int protocol_size);\ ijkav_register_##x##_protocol(&ijkimp_ff_##x##_protocol, sizeof(URLProtocol)); \ }
URLProtocol structure is the key. So what does the filling of this structure depend on? Look at the macro call, find the back part of extern, search, the original in many files, such as ijkurlhook.c file:
URLProtocol ijkimp_ff_ijktcphook_protocol = { .name = "ijktcphook", .url_open2 = ijktcphook_open, .url_read = ijkurlhook_read, .url_write = ijkurlhook_write, .url_close = ijkurlhook_close, .priv_data_size = sizeof(Context), .priv_data_class = &ijktcphook_context_class, };
Functions for opening, writing, closing, etc. have been specified here. At this point, the operations corresponding to the basic protocol are found. Let's look at different ways of handling live:
#define IJK_REGISTER_DEMUXER(x) \ { \ extern AVInputFormat ijkff_##x##_demuxer; \ ijkav_register_input_format(&ijkff_##x##_demuxer); \ }
It will then be located in the ijklivehook.c file
AVInputFormat ijkff_ijklivehook_demuxer = { .name = "ijklivehook", .long_name = "Live Hook Controller", .flags = AVFMT_NOFILE | AVFMT_TS_DISCONT, .priv_data_size = sizeof(Context), .read_probe = ijklivehook_probe, .read_header2 = ijklivehook_read_header, .read_packet = ijklivehook_read_packet, .read_close = ijklivehook_read_close, .priv_class = &ijklivehook_class, };
Looking down, take ijklivehook_read_header as an example, you can see that there is url judgment inside, distinguishing rtmp from rtsp, which is clear now.
To sum up, URLProtocol is used to normalize all protocols. Names and operation functions are defined here.
Back to ffp_global_init, we proceed to av_init_packet. Here's a data structure called AVPacket. This is a structure for storing information related to compressed coded data.
typedef struct AVPacket { /** * A reference to the reference-counted buffer where the packet data is * stored. * May be NULL, then the packet data is not reference-counted. */ AVBufferRef *buf; /** * Presentation timestamp in AVStream->time_base units; the time at which * the decompressed packet will be presented to the user. * Can be AV_NOPTS_VALUE if it is not stored in the file. * pts MUST be larger or equal to dts as presentation cannot happen before * decompression, unless one wants to view hex dumps. Some formats misuse * the terms dts and pts/cts to mean something different. Such timestamps * must be converted to true pts/dts before they are stored in AVPacket. */ int64_t pts; /** * Decompression timestamp in AVStream->time_base units; the time at which * the packet is decompressed. * Can be AV_NOPTS_VALUE if it is not stored in the file. */ int64_t dts; uint8_t *data; int size; int stream_index; /** * A combination of AV_PKT_FLAG values */ int flags; /** * Additional packet data that can be provided by the container. * Packet can contain several types of side information. */ AVPacketSideData *side_data; int side_data_elems; /** * Duration of this packet in AVStream->time_base units, 0 if unknown. * Equals next_pts - this_pts in presentation order. */ int64_t duration; int64_t pos; ///< byte position in stream, -1 if unknown #if FF_API_CONVERGENCE_DURATION /** * @deprecated Same as the duration field, but as int64_t. This was required * for Matroska subtitles, whose duration values could overflow when the * duration field was still an int. */ attribute_deprecated int64_t convergence_duration; #endif } AVPacket;
Did you see anything? pts,dts,data. Display timestamp, decode timestamp, data. Av_init_package is a simple filling, no code. Back to JNI_OnLoad, then ijkmp_global_set_inject_callback.
Set up a callback, then look at the specific callback agreement:
static int inject_callback(void *opaque, int what, void *data, size_t data_size) { JNIEnv *env = NULL; jobject jbundle = NULL; int ret = -1; SDL_JNI_SetupThreadEnv(&env); jobject weak_thiz = (jobject) opaque; if (weak_thiz == NULL ) goto fail; switch (what) { case AVAPP_CTRL_WILL_HTTP_OPEN: case AVAPP_CTRL_WILL_LIVE_OPEN: case AVAPP_CTRL_WILL_CONCAT_SEGMENT_OPEN: { AVAppIOControl *real_data = (AVAppIOControl *)data; real_data->is_handled = 0; jbundle = J4AC_Bundle__Bundle__catchAll(env); if (!jbundle) { ALOGE("%s: J4AC_Bundle__Bundle__catchAll failed for case %d\n", __func__, what); goto fail; } J4AC_Bundle__putString__withCString__catchAll(env, jbundle, "url", real_data->url); J4AC_Bundle__putInt__withCString__catchAll(env, jbundle, "segment_index", real_data->segment_index); J4AC_Bundle__putInt__withCString__catchAll(env, jbundle, "retry_counter", real_data->retry_counter); real_data->is_handled = J4AC_IjkMediaPlayer__onNativeInvoke(env, weak_thiz, what, jbundle); if (J4A_ExceptionCheck__catchAll(env)) { goto fail; } J4AC_Bundle__getString__withCString__asCBuffer(env, jbundle, "url", real_data->url, sizeof(real_data->url)); if (J4A_ExceptionCheck__catchAll(env)) { goto fail; } ret = 0; break; } case AVAPP_EVENT_WILL_HTTP_OPEN: case AVAPP_EVENT_DID_HTTP_OPEN: case AVAPP_EVENT_WILL_HTTP_SEEK: case AVAPP_EVENT_DID_HTTP_SEEK: { AVAppHttpEvent *real_data = (AVAppHttpEvent *) data; jbundle = J4AC_Bundle__Bundle__catchAll(env); if (!jbundle) { ALOGE("%s: J4AC_Bundle__Bundle__catchAll failed for case %d\n", __func__, what); goto fail; } J4AC_Bundle__putString__withCString__catchAll(env, jbundle, "url", real_data->url); J4AC_Bundle__putLong__withCString__catchAll(env, jbundle, "offset", real_data->offset); J4AC_Bundle__putInt__withCString__catchAll(env, jbundle, "error", real_data->error); J4AC_Bundle__putInt__withCString__catchAll(env, jbundle, "http_code", real_data->http_code); J4AC_IjkMediaPlayer__onNativeInvoke(env, weak_thiz, what, jbundle); if (J4A_ExceptionCheck__catchAll(env)) goto fail; ret = 0; break; } case AVAPP_CTRL_DID_TCP_OPEN: case AVAPP_CTRL_WILL_TCP_OPEN: { AVAppTcpIOControl *real_data = (AVAppTcpIOControl *)data; jbundle = J4AC_Bundle__Bundle__catchAll(env); if (!jbundle) { ALOGE("%s: J4AC_Bundle__Bundle__catchAll failed for case %d\n", __func__, what); goto fail; } J4AC_Bundle__putInt__withCString__catchAll(env, jbundle, "error", real_data->error); J4AC_Bundle__putInt__withCString__catchAll(env, jbundle, "family", real_data->family); J4AC_Bundle__putString__withCString__catchAll(env, jbundle, "ip", real_data->ip); J4AC_Bundle__putInt__withCString__catchAll(env, jbundle, "port", real_data->port); J4AC_Bundle__putInt__withCString__catchAll(env, jbundle, "fd", real_data->fd); J4AC_IjkMediaPlayer__onNativeInvoke(env, weak_thiz, what, jbundle); if (J4A_ExceptionCheck__catchAll(env)) goto fail; ret = 0; break; } default: { ret = 0; } } fail: SDL_JNI_DeleteLocalRefP(env, &jbundle); return ret; }
Look at a simple function: J4AC_IjkMediaPlayer__onNativeInvoke, found a definition in the java layer:
private OnNativeInvokeListener mOnNativeInvokeListener; public void setOnNativeInvokeListener(OnNativeInvokeListener listener) { mOnNativeInvokeListener = listener; } public interface OnNativeInvokeListener { int CTRL_WILL_TCP_OPEN = 0x20001; // NO ARGS int CTRL_DID_TCP_OPEN = 0x20002; // ARG_ERROR, ARG_FAMILIY, ARG_IP, ARG_PORT, ARG_FD int CTRL_WILL_HTTP_OPEN = 0x20003; // ARG_URL, ARG_SEGMENT_INDEX, ARG_RETRY_COUNTER int CTRL_WILL_LIVE_OPEN = 0x20005; // ARG_URL, ARG_RETRY_COUNTER int CTRL_WILL_CONCAT_RESOLVE_SEGMENT = 0x20007; // ARG_URL, ARG_SEGMENT_INDEX, ARG_RETRY_COUNTER int EVENT_WILL_HTTP_OPEN = 0x1; // ARG_URL int EVENT_DID_HTTP_OPEN = 0x2; // ARG_URL, ARG_ERROR, ARG_HTTP_CODE int EVENT_WILL_HTTP_SEEK = 0x3; // ARG_URL, ARG_OFFSET int EVENT_DID_HTTP_SEEK = 0x4; // ARG_URL, ARG_OFFSET, ARG_ERROR, ARG_HTTP_CODE String ARG_URL = "url"; String ARG_SEGMENT_INDEX = "segment_index"; String ARG_RETRY_COUNTER = "retry_counter"; String ARG_ERROR = "error"; String ARG_FAMILIY = "family"; String ARG_IP = "ip"; String ARG_PORT = "port"; String ARG_FD = "fd"; String ARG_OFFSET = "offset"; String ARG_HTTP_CODE = "http_code"; /* * @return true if invoke is handled * @throws Exception on any error */ boolean onNativeInvoke(int what, Bundle args); } @CalledByNative private static boolean onNativeInvoke(Object weakThiz, int what, Bundle args) { DebugLog.ifmt(TAG, "onNativeInvoke %d", what); if (weakThiz == null || !(weakThiz instanceof WeakReference<?>)) throw new IllegalStateException("<null weakThiz>.onNativeInvoke()"); @SuppressWarnings("unchecked") WeakReference<IjkMediaPlayer> weakPlayer = (WeakReference<IjkMediaPlayer>) weakThiz; IjkMediaPlayer player = weakPlayer.get(); if (player == null) throw new IllegalStateException("<null weakPlayer>.onNativeInvoke()"); OnNativeInvokeListener listener = player.mOnNativeInvokeListener; if (listener != null && listener.onNativeInvoke(what, args)) return true; switch (what) { case OnNativeInvokeListener.CTRL_WILL_CONCAT_RESOLVE_SEGMENT: { OnControlMessageListener onControlMessageListener = player.mOnControlMessageListener; if (onControlMessageListener == null) return false; int segmentIndex = args.getInt(OnNativeInvokeListener.ARG_SEGMENT_INDEX, -1); if (segmentIndex < 0) throw new InvalidParameterException("onNativeInvoke(invalid segment index)"); String newUrl = onControlMessageListener.onControlResolveSegmentUrl(segmentIndex); if (newUrl == null) throw new RuntimeException(new IOException("onNativeInvoke() = <NULL newUrl>")); args.putString(OnNativeInvokeListener.ARG_URL, newUrl); return true; } default: return false; } }
Then you can make sure that this is a registered callback to notify the java layer. Well, back to JNI_OnLoad, it's FFmpegApi_global_init:
#define JNI_CLASS_FFMPEG_API "tv/danmaku/ijk/media/player/ffmpeg/FFmpegApi" ...... int FFmpegApi_global_init(JNIEnv *env) { int ret = 0; IJK_FIND_JAVA_CLASS(env, g_clazz.clazz, JNI_CLASS_FFMPEG_API); (*env)->RegisterNatives(env, g_clazz.clazz, g_methods, NELEM(g_methods)); return ret; }
By definition, there is only one sentence to find this class:
public class FFmpegApi { public static native String av_base64_encode(byte in[]); }
Actually, it's a base64 decoder, pointing to the c function of ffmpeg, and registered here.
Finally, the analysis is finished, summed up is all kinds of initialization, protocol, decoder, network, callback upper layer.