Replacement Kernel
[Non-native Replacement Kernel] In this paper, the compiler and compiler modules are separated, but the principle and result are the same. 1) Compile the kernel: #make bzImage When the kernel is compiled successfully, a new kernel image file bzImage is generated in the source tree root directory, linux-3.12.5/arch/x86_64/boot/directory. Note: For large kernels (such as SCSI support), make bzImage (recommended) For small kernels, make zImage. 2) Compiler module: #make modules Compile loadable modules (i.e. the option of selecting M in the kernel options) for future loading using the insmod command. Compilation time depends on the number of M options. 3) Installation module: #make modules_install Copy the compiled modules to / lib/modules. 4) Install the kernel: Because it's not a native replacement kernel, we need to copy the newly generated kernel files to the machine to replace the kernel. There are four main kernel files to be copied, namely. config, bzImage, System.map and system / lib/modules/linux-3.12.5 directories. The specific operation is as follows: (1) Copy the generated linux-3.12.5/.config to / boot to replace the kernel system and rename it config-3.12.5 (2) Copy the generated linux-3.12.5/arch/x86-64/boot/bzImage to replace the kernel system/boot and rename it vmlinuz-3.12.5 (Note: Special attention should be paid to changing the name of the copied file to vmlinuz-x.x.x). (3) Copy the generated linux-3.12.5/System.map to / boot to replace the kernel system and rename it System.map-3.12.5 (4) Copy the system directory / lib/modules/linux-3.12.5 generated by make modules_install to / lib/modules to replace the kernel system. (5) Execute the following commands in the / lib/modules directory of the system to replace the new kernel: Sudo update-initramfs-c-k4.4.179-rk3399, generating/boot/initrd.img-4.4.179-rk3399 sudo update-grub (or # update-grub2), update the boot load file.
make ARCH=arm64 nanopi4-images -j20 sudo make install
FriendlyArm calls mipi scripts
cd /tmp/ git clone https://github.com/friendlyarm/gst-camera-sh.git sudo cp gst-camera-sh/*.sh /usr/bin/
$ gst-camera.sh Start MIPI CSI Camera Preview [/dev/video0] ... XCAM INFO device_manager.cpp:90: open /usr/lib/libcam_ia.so successed XCAM INFO device_manager.cpp:96: open /usr/lib/libcam_engine_cifisp.so successed XCAM INFO device_manager.cpp:106: dlsym rkisp_start success Setting pipeline to PAUSED ... XCAM WARNING v4l2_device.cpp:171: set capture mode(0x00004000) failed but continue XCAM INFO x3a_image_process_center.cpp:48: Add processor(IspImageProcessor) into image processor center Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock XCAM INFO v4l2_device.cpp:322: device(/dev/video0) set format(w:1280, h:720, pixelformat:NV12, bytesperline:1280) XCAM WARNING rkisp_device.cpp:48: atomisp device(/dev/video0) get framerate failed XCAM ERROR rkisp_device.cpp:72: atomisp device(/dev/video0) try subdev format failed XCAM WARNING v4l2_device.cpp:233: device(/dev/video0) pre_set_format failed XCAM INFO v4l2_device.cpp:300: device(/dev/video0) set format(w:1280, h:720, pixelformat:NV12, bytesperline:1280,image_size:1382400) XCAM INFO device_manager.cpp:208: device manager start, capture dev fd: 10 CAMHALInfo: CAMHALVERSION is: 2.0.9.0 CAMHALInfo: libcam_ia: 2.0.c.0 CAMHALInfo: init:tuningFile /etc/cam_iq/ov13850.xml CAMHALInfo: load tunning file success. AWB: AwbConfigure: AWB_VERSION_10 CAMHALInfo: rkisp_start: interface isp dev started XCAM INFO device_manager.cpp:211: device manager isp_init _rkisp_engine:0x7f8c0020b0 XCAM INFO device_manager.cpp:216: rkisp_init engine succeed XCAM INFO v4l2_device.cpp:473: request buffers in device(/dev/video0) XCAM INFO v4l2_device.cpp:563: init buffer poll in device(/dev/video0), cnt: 8 XCAM INFO v4l2_device.cpp:440: device(/dev/video0) started successfully XCAM INFO xcam_analyzer.cpp:164: Analyzer(X3aAnalyzerSimple) initialized(w:1280, h:720). XCAM INFO xcam_analyzer.cpp:213: Analyzer(X3aAnalyzerSimple) started in async mode. XCAM INFO image_processor.cpp:182: ImageProcessor(IspImageProcessor) started XCAM INFO x3a_image_process_center.cpp:84: 3a process center started AWB: AwbConfigure: AWB_VERSION_10 =================================================================================== handling interrupt. Interrupt: Stopping pipeline ... Execution ended after 0:04:35.964235671 Setting pipeline to PAUSED ... Setting pipeline to READY ... XCAM INFO xcam_analyzer.cpp:226: Analyzer(X3aAnalyzerSimple) stopped. XCAM INFO xcam_analyzer.cpp:178: Analyzer(X3aAnalyzerSimple) deinited. XCAM INFO device_manager.cpp:343: deinit rkisp engine CAMHALInfo: rkisp_stop: rkisp interface ready to deinit XCAM INFO v4l2_device.cpp:457: device(/dev/video0) stopped Setting pipeline to NULL ... Freeing pipeline ...
Complete command:
gst-launch-1.0 rkisp device=/dev/video1 io-mode=4 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! rkximagesink
$ vim gst-camera.sh
if [ ${CameraTypes[$icam]} = "mipi" ]; then
CMD="gst-launch-1.0 rkisp ${rkargs} io-mode=4 \
! video/x-raw,format=NV12,${PreviewModes[$icam]} \
! ${Sinks[$icam]}"
Install GStreamer
sudo apt install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio
Error:
no element "rkisp" No element "rkisp", no plug-in, named libgsrkisp.so
https://github.com/rockchip-linux/camera_engine_rkisp
$ vim productConfigs.mk line 16 CROSS_COMPILE ?= /usr/bin/ $ sudo make ARCH=arm
./autogen.sh –prefix=/home/install –nocheck fatal: Not a git repository (or any of the parent directories): .git ln: failed to create symbolic link '.git/hooks/pre-commit': No such file or directory + passing argument –prefix=/home/install to configure + autotools version check disabled + options passed to configure: –prefix=/home/install + check for build tools: skipped version checks patching file po/Makefile.in.in Hunk #1 succeeded at 15 with fuzz 2 (offset 4 lines). + running autopoint –force... ./autogen.sh: 349: ./autogen.sh: autopoint: not found //This problem is mainly due to the unsuccessful installation of auto-related tools. //Just install with apt install sudo apt install autoconf automake libtool autopoint
configure: No package 'glib-2.0' found No package 'gobject-2.0' found No package 'gthread-2.0' found No package 'gmodule-no-export-2.0' found configure: error: This package requires GLib >= 2.20 to compile. sudo apt-get install libperl-dev libgtk2.0-dev libgstreamer-plugins-base1.0-dev libgstreamer1.0-dev
https://forum.armbian.com/topic/9528-nano-pi-m4-camera-module/
If you want to get your hands dirty i can give you briefly the steps if i recall correctly (it works on NanoPi M4, so should work on NanoPi-T4) but unfortunately, i did not take note: a) encode/decode is done by gstreamer (decoding by mvp) install gstreamer (if not already installed) sudo apt install libgstreamer1.0-0 gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-libav gstreamer1.0-doc gstreamer1.0-tools gstreamer1.0-x gstreamer1.0-alsa gstreamer1.0-gl gstreamer1.0-gtk3 gstreamer1.0-qt5 gstreamer1.0-pulseaudio b) clone https://github.com/rockchip-linux/mpp , build and install it (install dependencies fisrt) cmake -DRKPLATFORM=ON -DHAVE_DRM=ON && make sudo make install c) clone gstreamer-rockchip (i can't remember if i did for the -extra also) git clone https://github.com/rockchip-linux/gstreamer-rockchip.git cd gstreamer-rockchip ./autogen.sh --disable-rkximage make sudo make install sudo ldconfig vim ./gst/rkximage/rkx_kmsutils.c:50:3 git clone https://github.com/rockchip-linux/gstreamer-rockchip-extra sudo ./autogen.sh sudo make sudo make install sudo ldconfig d) check if the plugins are then installed correctly gst-inspect-1.0|grep 4l2 gst-inspect-1.0|grep rkcamsrc gst-inspect-1.0 | grep rk gst-inspect-1.0 rkximage gst-inspect-1.0 rkv4l2 You will need libdrm-dev and some other dependencies i can't remember. This steps should work on any RK3399. Have fun. vim ./gst/kms/gstkmsutils.c
MIPI OV13850
GStreamer document
https://gstreamer.freedesktop.org/documentation/installing/index.html?gi-language=c#
Summary of element
Previous articles introduced v4l2src, video decoder coding, gst-omx and videosink display several elements in detail, and introduced the operation process of their interaction with other elements. This time, we summarized the previous articles.
element Creation
_Since GStreamer initially scans and loads libraries in the / usr/lib/gstreamer-1.0 directory, identifies feature s and records relevant information. When a function is used, check whether gstreamer core supports it. If so, load the corresponding library, get information, and create the corresponding element instance. In this way, the element is created.
element link
After creation, it will be added to bin and link element, and when it passes through link, it will call the query function of the corresponding element through gst_pad_query_caps (pad, NULL). This query function will call the init() function of element through the pad query function set by gst_pad_set_query_function(), which generally returns the pad template caps at the element link stage. Because this stage has not yet carried out opening devices, device initialization and other operations, do not know the elements really support caps, but element link stage as long as the query caps have intersection can link successfully, so the problem is not big.
NULL->READY
_In this state, the corresponding device initialization will be carried out, the corresponding hardware device will be initialized by calling the start() function or open() function according to the corresponding class, and the class structure parameters will be initialized.
READY->PAUSED
_This step will further apply for resources and determine the appropriate parameter settings. An important step will be to activate pad. The activation process will call the pad activation function set by gst_pad_set_activate_function() in element init. At the same time, the corresponding push mode will be selected, so the pad activation process will be called. Call the pad activation mode function set by element init through gst_pad_set_activatemode_function().
_When pad is activated, the GST_PAD_FLAG_NEED_RECONFIGURE flag will be set up in general. There will be checking this flag in element, and then negotiation will be carried out. In the process of negotiation, the pad query function of element will be called many times. Why is it called many times? At the beginning of link, it will be called. The result is pad template caps. After device initialization, the actual hardware will be queried and the caps that are really supported will be obtained. In this way, the upstream will know the caps that are supported downstream.
After caps, it will receive stream-start events. The upstream informs the element that it will start transmitting data. Who will handle EVENT? In general, the EVENT processing function of pad is set up in element init through gst_pad_set_event_function(). Element receives the event and extracts the corresponding information initialization structure.
_Although the above feedback caps to the upstream, upstream according to their own situation to select the appropriate caps, and then will send accept-caps query, again query, whether element ary support the caps.
When the upstream receives feedback again, the caps event is sent to the element, and the element receives the event and sets_caps. At the same time, upstream will send GST_QUERY_ALLOCATION to query the attributes of buffer pool, determine the attributes of buffer pool, decide when to apply for buffer pool, and activate pool.
Finally, GST_EVENT_SEGMENT and TAG EVENT. Segments will describe the next data timestamp range, and subsequent data will need to be in this range for normal use. TAG is the other information describing the data, these, element event processing functions will have corresponding operations.
The next step is data preroll. When the data arrives, the chain function set by gst_pad_set_chain_function() in element init will be called. Check whether the timestamp of the data is in the segment, data synchronization, roll-ahead through the preroll function, and finally commit status, enter PAUSED.
PAUSED->PLAYING
In this process, the clock will be set, the clock will run, when the chain function receives the data, check the time validity, synchronize and process the data, push to the downstream, send QOS events to the upstream, and complete a cycle.
summary
_ Of course, the best way is to analyze the specific process by capturing the running log information.
Common commands of Gstreamer
#Used to check all attributes, signals, dynamic parameters and object hierarchies of elements $ gst-inspect-1.0 #gst-launch-1.0 is a simple script-like command-line application that can be used to test pipelines. $ gst-launch-1.0 gst-launch-1.0 rkisp device=/dev/video1 io-mode=4 ! video/x-raw,format=NV12,width=1280,height=720,framerate=30/1 ! kmssink
rockchip-isp1
http://blog.iotwrt.com/media/2017/10/01/camera/
The original FriendlyElec release used gstreamer-camera instead of gstreamer-plugins/extra-I hope-that's one trick: (from github.com/rockchip-linux/repos) git clone https://github.com/rockchip-linux/gstreamer-camera sudo ./autogen sudo make sudo make install
View Plug-in Scan Path $ echo $GST_PLUGIN_PATH
#View Plug-in Information
$ gst-inspect-1.0 audiotestsrc
Factory Details:
Rank none (0)
Long-name Audio test source
Klass Source/Audio
Description Creates audio test signals of given frequency and volume
Author Stefan Kost <ensonic@users.sf.net>
Plugin Details:
Name audiotestsrc
Description Creates audio test signals of given frequency and volume
Filename /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstaudiotestsrc.so
Version 1.14.4
License LGPL
Source module gst-plugins-base
Source release date 2018-10-02
Binary package GStreamer Base Plugins (Ubuntu)
Origin URL https://launchpad.net/distros/ubuntu/+source/gst-plugins-base1.0
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstBaseSrc
+----GstAudioTestSrc
Pad Templates:
SRC template: 'src'
Availability: Always
Capabilities:
audio/x-raw
format: { (string)S16LE, (string)S16BE, (string)U16LE, (string)U16BE, (string)S24_32LE, (string)S24_32BE, (string)U24_32LE, (string)U24_32BE, (string)S32LE, (string)S32BE, (string)U32LE, (string)U32BE, (string)S24LE, (string)S24BE, (string)U24LE, (string)U24BE, (string)S20LE, (string)S20BE, (string)U20LE, (string)U20BE, (string)S18LE, (string)S18BE, (string)U18LE, (string)U18BE, (string)F32LE, (string)F32BE, (string)F64LE, (string)F64BE, (string)S8, (string)U8 }
layout: interleaved
rate: [ 1, 2147483647 ]
channels: [ 1, 2147483647 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SRC: 'src'
Pad Template: 'src'
Element Properties:
name : The name of the object
flags: readable, writable
String. Default: "audiotestsrc0"
parent : The parent of the object
flags: readable, writable
Object of type "GstObject"
blocksize : Size in bytes to read per buffer (-1 = default)
flags: readable, writable
Unsigned Integer. Range: 0 - 4294967295 Default: 4294967295
num-buffers : Number of buffers to output before sending EOS (-1 = unlimited)
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: -1
typefind : Run typefind before negotiating (deprecated, non-functional)
flags: readable, writable, deprecated
Boolean. Default: false
do-timestamp : Apply current stream time to buffers
flags: readable, writable
Boolean. Default: false
samplesperbuffer : Number of samples in each outgoing buffer
flags: readable, writable
Integer. Range: 1 - 2147483647 Default: 1024
wave : Oscillator waveform
flags: readable, writable, controllable
Enum "GstAudioTestSrcWave" Default: 0, "sine"
(0): sine - Sine
(1): square - Square
(2): saw - Saw
(3): triangle - Triangle
(4): silence - Silence
(5): white-noise - White uniform noise
(6): pink-noise - Pink noise
(7): sine-table - Sine table
(8): ticks - Periodic Ticks
(9): gaussian-noise - White Gaussian noise
(10): red-noise - Red (brownian) noise
(11): blue-noise - Blue noise
(12): violet-noise - Violet noise
freq : Frequency of test signal. The sample rate needs to be at least 4 times higher.
flags: readable, writable, controllable
Double. Range: 0 - 5.368709e+08 Default: 440
volume : Volume of test signal
flags: readable, writable, controllable
Double. Range: 0 - 1 Default: 0.8
is-live : Whether to act as a live source
flags: readable, writable
Boolean. Default: false
timestamp-offset : An offset added to timestamps set on buffers (in ns)
flags: readable, writable
Integer64. Range: -9223372036854775808 - 9223372036854775807 Default: 0
can-activate-push : Can activate in push mode
flags: readable, writable
Boolean. Default: true
can-activate-pull : Can activate in pull mode
flags: readable, writable
Boolean. Default: falseFrom the above information, the plug-in installation path can be obtained: / usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstaudiotestsrc.so
#View Plug-in Information $ gst-inspect-1.0 ... ... Total count: 249 plugins (3 blacklist entries not shown), 1398 features //There are three plug-ins on the blacklist #View Blacklist Plugins $ gst-inspect-1.0 -b Blacklisted files: libgstkmssink.so librockchip_vpu.so librockchip_mpp.so Total count: 3 blacklisted files $ cd ~/.cache/gstreamer-1.0 $ sudo rm registry.aarch64.bin $ sudo GST_DEBUG=2,GST_PLUGIN_LOADING:5,GST_REGISTRY:5 gst-inspect-1.0 #Verify plug-in information gst-inspect-1.0 ./libgstkmssink.so #If you can find information, move the plug-in to / usr/lib/aarch64-linux-gnu/gstreamer-1.0 / directory #If not, move the plug-in file to / usr/lib/aarch64-linux-gnu / directory
#After installing the rkisp plug-in, execute sudo gst-camera.sh //Report errors: (gst-launch-1.0:5411): GStreamer-WARNING **: 07:18:44.346: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstkms.so': /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstkms.so: undefined symbol: gst_video_overlay_install_properties WARNING: erroneous pipeline: no element "kmssink"
https://github.com/rockchip-linux/gstreamer-rockchip/issues/12