I haven't written java for a long time. I came back this time because the voice function is required before the project. Turing connected with the company only supports amr and opus audio format upload for recognition, However, as far as I know, the microphone provided by unity only supports wav and mp3. Therefore, the previous solution has been to first transmit audio data through Baidu api that can recognize wav format, analyze and return speech recognition results, and then send the results back to Turing's api interface in the form of text, So as to indirectly realize the function of voice dialogue. However, it is obvious that there are many disadvantages. First, it needs additional workload and energy to access Baidu. Second, a voice function needs the api of two different platforms to realize. Obviously, it is not desirable, but it can only be realized in this way due to the time and personal ability at that time
Now, with continuous learning and clear logical thinking of the code, and the previous use of Baidu's voice wake-up function, I have a deeper understanding of Android code. Therefore, I want to challenge the direct return of amr and opus format audio files to Turing interface through Android's native microphone, so that the steps of voice recognition through Baidu can be removed
Try turning on the microphone
First, you need to create a new project in as, and then create a new library
Import the unity-classes.jar file into libs, and then reference it
For the specific steps of opening the microphone, please refer to the following post. I basically follow the cat and draw the tiger
https://www.cnblogs.com/vir56k/archive/2012/11/29/2794226.html
You need to pay attention here
Due to the update of Android, different versions of Android have different processing methods on authorization on different mobile terminals
For Android 6.0 or above, you need to apply for permission dynamically when the program is running
Here you can refer to the posts of other big guys who dynamically apply for permission. My little brother suddenly can't find the website of the big guy's post after a day
Oh, I remember that the method of dynamically applying for permission is from the post I read when using Baidu voice wake-up. I did this when initializing voice wake-up and recognition in the boss's code. Then I found that he added a Context to the formal parameter of the method of initializing dynamic application permission. I clicked in and looked at it if I didn't know java, I found that this is actually an activity, and then I used the idea mentioned above to pass the activity of unit back to the method of initializing abortion application permission in aar package
Here I post the logic of some changes I made after reading it
Here is the java code
public String startRecording() {
mediaRecorder = new MediaRecorder();
mediaRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
mediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.AMR_NB);
mediaRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
mediaRecorder.setOutputFile(newFileName());
try {
mediaRecorder.prepare();
} catch (Exception e) {
Log.e("test", e.getMessage());
}
mediaRecorder.start();
return "finied";
}
public String newFileName() {
//String mFileName = Environment.getExternalStorageDirectory() + File.separator + Environment.DIRECTORY_DCIM + File.separator;
String mFileName = Environment.getExternalStorageDirectory()
.getAbsolutePath();
String s = new SimpleDateFormat("yyyy-MM-dd hhmmss")
.format(new Date());
mFileName += "/rcd_" + s + ".amr";
Log.d("test", mFileName);
return mFileName;
}
/**
* android 6.0 The above requires dynamic application permission
*/
public void initPermission(Context context) {
String permissions[] = {Manifest.permission.RECORD_AUDIO,
Manifest.permission.ACCESS_NETWORK_STATE,
Manifest.permission.INTERNET,
Manifest.permission.WRITE_EXTERNAL_STORAGE
};
PackageManager pm = getActivityByContext(context).getPackageManager();
boolean permission_readStorage = (PackageManager.PERMISSION_GRANTED ==
pm.checkPermission("android.permission.RECORD_AUDIO", "com.cientx.tianguo"));
boolean permission_network_state = (PackageManager.PERMISSION_GRANTED ==
pm.checkPermission("android.permission.ACCESS_NETWORK_STATE", "com.cientx.tianguo"));
boolean permission_internet = (PackageManager.PERMISSION_GRANTED ==
pm.checkPermission("android.permission.INTERNET", "com.cientx.tianguo"));
boolean permission_writeStorage = (PackageManager.PERMISSION_GRANTED ==
pm.checkPermission("android.permission.WRITE_EXTERNAL_STORAGE", "com.cientx.tianguo"));
if (!(permission_readStorage && permission_writeStorage && permission_network_state && permission_internet)) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
getActivityByContext(context).requestPermissions(permissions, 0x01);
}
}
}
However, it should be noted that activity is used in this boss's post, but we are in the unit project. We don't need to make another activity in Android. What should we do? Chicken thief, I thought of a method, because there was no independent activity in Baidu's voice wake-up function before, But they can still get the activity in the same way as the above boss, so I went to the source code of Baidu voice wake-up post and found that the boss's implementation method is to call the AndroidJavaClass class class in the unit, then pass the path of the unit, take the activity of the project as the activity, and then pass it back to the aar package
Here I can only say nb, big guy or big guy
Take the unit as the activity and then send it back to the method of initializing the dynamic permission application in the aar package
This is a difficult point about the method of obtaining permissions dynamically
The second difficulty is about storing files
As shown below
The file storage path needs to be set during microphone initialization
As for the stored permissions, it seems that there will be problems only through the above dynamic application for permissions
For example, after using the above operation, I found that I couldn't call the microphone correctly. The debug found that it was because
/storage/emulated/0/rcd_2021-09-11 094229.amr: open failed: EPERM (Operation
After thinking about this exception for a long time in Baidu and, I found that the way to obtain the permission to read and write files also has different processing methods in different Android versions (it's really outrageous. Open the fucking door to outrageous - outrageous home)
In addition to the above dynamic permission application methods, you also need to add the following tag in the AndroidManifest.xml file
<application android:requestLegacyExternalStorage="true" />
See this post and comments (mainly comments) for details
https://blog.csdn.net/qq_34884729/article/details/53284274
There are two ways to package, one is aar and the other is jar
Note that you can only play aar bags here (from my current ability)
This is because if the jar package is typed, the AndroidManifest.xml file will not be typed, that is to say, the permission tags in it will be gone, so this method is not advisable
jar package is more suitable for some functions that do not require permission (personal feeling)
Delete unity-classes.jar from aar and import it into unity (the old question)
Just call the corresponding method in unity
Post the complete code~
This is a single example (in the post of the big man who was awakened by Baidu voice)
He encapsulates all functions in this singleton, and then calls the required methods directly through this singleton
import android.content.Context;
public class CientBaiDuVoiceMainActivity {
public static CientBaiDuVoiceMainActivity _instance;
public static CientBaiDuVoiceMainActivity getInstance() {
if (_instance == null) {
_instance = new CientBaiDuVoiceMainActivity();
}
return _instance;
}
MroTest mroTest;
public void InitMroTest(Context context){
try{
mroTest = new MroTest();
}catch (Exception e){
System.out.println(e.getMessage());
}
mroTest.initPermission(context);
}
}
This is my own microphone management class after finishing the posts of Baidu voice wake-up boss and other online posts
import android.Manifest;
import android.content.Context;
import android.content.pm.PackageManager;
import android.media.MediaRecorder;
import android.os.Build;
import android.os.Environment;
import android.util.Log;
import java.text.SimpleDateFormat;
import java.util.Date;
import static com.pure.testmcro.GetActivity.getActivityByContext;
public class MroTest {
public MediaRecorder mediaRecorder;
public String startRecording() {
mediaRecorder = new MediaRecorder();
mediaRecorder.setAudioSource(MediaRecorder.AudioSource.MIC);
mediaRecorder.setOutputFormat(MediaRecorder.OutputFormat.AMR_NB);
mediaRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AMR_NB);
mediaRecorder.setOutputFile(newFileName());
try {
mediaRecorder.prepare();
} catch (Exception e) {
Log.e("test", e.getMessage());
}
mediaRecorder.start();
return "finied";
}
public String newFileName() {
//String mFileName = Environment.getExternalStorageDirectory() + File.separator + Environment.DIRECTORY_DCIM + File.separator;
String mFileName = Environment.getExternalStorageDirectory()
.getAbsolutePath();
String s = new SimpleDateFormat("yyyy-MM-dd hhmmss")
.format(new Date());
mFileName += "/rcd_" + s + ".amr";
Log.d("test", mFileName);
return mFileName;
}
public void stopRecording() {
mediaRecorder.stop();
mediaRecorder.release();
mediaRecorder = null;
}
public void test() {
Log.d("pure", "test: test");
}
/**
* android 6.0 The above requires dynamic application permission
*/
public void initPermission(Context context) {
String permissions[] = {Manifest.permission.RECORD_AUDIO,
Manifest.permission.ACCESS_NETWORK_STATE,
Manifest.permission.INTERNET,
Manifest.permission.WRITE_EXTERNAL_STORAGE
};
PackageManager pm = getActivityByContext(context).getPackageManager();
boolean permission_readStorage = (PackageManager.PERMISSION_GRANTED ==
pm.checkPermission("android.permission.RECORD_AUDIO", "com.cientx.tianguo"));
boolean permission_network_state = (PackageManager.PERMISSION_GRANTED ==
pm.checkPermission("android.permission.ACCESS_NETWORK_STATE", "com.cientx.tianguo"));
boolean permission_internet = (PackageManager.PERMISSION_GRANTED ==
pm.checkPermission("android.permission.INTERNET", "com.cientx.tianguo"));
boolean permission_writeStorage = (PackageManager.PERMISSION_GRANTED ==
pm.checkPermission("android.permission.WRITE_EXTERNAL_STORAGE", "com.cientx.tianguo"));
if (!(permission_readStorage && permission_writeStorage && permission_network_state && permission_internet)) {
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
getActivityByContext(context).requestPermissions(permissions, 0x01);
}
}
}
}
There is also a tool class
import android.app.Activity;
import android.content.Context;
import android.content.ContextWrapper;
public class GetActivity {
public static Activity getActivityByContext(Context context){
while(context instanceof ContextWrapper){
if(context instanceof Activity){
return (Activity) context;
}
context = ((ContextWrapper) context).getBaseContext();
}
return null;
}
}
Then there is the Android manifest file (Baidu voice is also used here to wake up the boss, so some don't need to be used. Look at the deletion)
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools"
package="com.pure.testmcro">
<!-- General authority -->
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<!-- Speech recognition permission -->
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<!-- Voice synthesis permission -->
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
<uses-permission android:name="android.permission.WRITE_SETTINGS"
tools:ignore="ProtectedPermissions" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<application android:requestLegacyExternalStorage="true" />
</manifest>
I also post the code for packaging aar and jar (Baidu search actually has it). Note that the packaged code needs to be placed in build and gradle
task CopyPlugin(type: Copy) {
dependsOn assemble
from('build/outputs/aar')
into('../../Assets/Plugins/Android')
include(project.name + '-release.aar')
}
//It's packaged into aar
///--------------Split line---------------------
//The following is packaged into jar
task deleteOldJar(type: Delete) {
delete 'release/AndroidPlugin.jar'
}
//task to export contents as jar
task exportJar(type: Copy) {
from('build/intermediates/packaged-classes/release/') //Take out the default jar package from this directory
into('release/')
include('classes.jar')
///Rename the jar
rename('classes.jar', 'AndroidPlugin.jar')
}
exportJar.dependsOn(deleteOldJar, build)
Then there is the code in unity
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
public class TextMcro : MonoBehaviour
{
AndroidJavaObject m_AndroidPluginObj;
AndroidJavaClass _androidJC;
AndroidJavaObject m_Android;
AndroidJavaClass jc;
AndroidJavaObject test1;
// Start is called before the first frame update
void Start()
{
}
// Update is called once per frame
void Update()
{
}
public void StartRecoding()
{
test1 = new AndroidJavaObject("com.pure.testmcro.MroTest");
string result = test1.Call<string>("startRecording");
Debug.Log(result);
}
public void EndRecoding()
{
test1 = new AndroidJavaObject("com.pure.testmcro.MroTest");
test1.Call("stopRecording");
}
public void test()
{
test1 = new AndroidJavaObject("com.pure.testmcro.MroTest");
test1.Call("test");
}
private void Awake()
{
_androidJC = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
if (_androidJC == null)
{
Debug.Log("JNI initialization failure.");
return;
}
m_AndroidPluginObj = _androidJC.GetStatic<AndroidJavaObject>("currentActivity");
jc = new AndroidJavaClass("com.pure.testmcro.CientBaiDuVoiceMainActivity");
m_Android = jc.CallStatic<AndroidJavaObject>("getInstance");
Debug.Log("Successful interaction with Android!");
if (m_Android == null)
{
Debug.Log("BaiduWake class ini failure");
}
InitREe();
}
public void InitREe()
{
if (m_Android != null)
{
m_Android.Call("InitMroTest", m_AndroidPluginObj);
Debug.Log("Wake up initialization succeeded!");
}
else
Debug.Log("AndroidPlugin is Null");
}
}
Mount the start recording and stop correcting buttons in the scene, and then package them
At present, the startrecoding method can be called perfectly, and the corresponding file can be found in memory. That is, the exception will be called when the stopcorrecting method is used
This problem hasn't been solved yet, mainly because I'm afraid I'll forget the previous ones, so I'll record it first