0.Integration preparation
Get sample code
Supported operating environment
- Windows 7 and above
- Linux (glibc minimum version 2.12)
- Mac
Authorized account
- AppKey and SecretKey
- Developer certificate aiengine.provision
SDK FILES
Download
- Interface file：AIEngine.java
- Library fileaiengine.dll (for Windows) libaiengine.so (for linux) libaiengine.dylib( for Mac)
Server address
- wss://cloud.chivox.com:443
Integrate SDK in the project
- Copy
aiengine.dll(libaiengine.so or libaiengine.dylib) under the SDK folder to the syspath directory of the Java project (e.g., the root of the project);
- Copy the developer certificate
aiengine.provision to the Java project's assets directory;
As shown in the figure below：
Overall process
1.Create an engine
1.1 Function prototype
- public static native long aiengine_new(String cfg, Object Context);
1.2 Function
- Create an instance of the engine, create a global evaluation engine when the product starts or enters the evaluation module, and subsequent evaluations can reuse the engine.
1.3 Parameter description
| Argument |
Description |
| Cfg |
Engine-related configuration, JSON format, should include appKey, secretKey, provision and other information. |
1.4 Cfg example
{
"appKey": "1351742***", //Required
"secretKey": "a3bed5523bbc020cf4a****", //Required
"provision": "path-of-provision-profile", //Required, development certificate storage path
"cloud": {
"server": "wss://cloud.chivox.com:443", // required, cloud service address
"connectTimeout": 20, //Optional, the default is 20s, the timeout period for establishing a network connection, the time is calculated from when the interface is called
"serverTimeout": 60 //Optional, the default is 60s, the response timeout time, the time starts from stopping the engine
},
"vad": { //Optional, voice activity detection
"enable": 1, //Optional, default 0. Set to 1 means that the engine created this time loads the voice activity detection function module. Setting it to 0 means that the engine created this time does not load the voice activity detection function.
"res": "./assets/vad.0.13.bin" //optional, vad resource path
"sampleRate": 16000, //optional, audio sampling rate, unit is Hz
"strip": 0 //Optional, whether to cut off the leading and trailing blanks when transmitting the audio data to the upper layer, generally set to 0
},
"prof": { //Debug function, optional
"enable": 0, //Debugging switch, disabled by default. Generally open during development and debugging stage, it is recommended to close before the product goes online
"output": "log-file-path" //If the debugging function is enabled, it must be selected, the debug log path, after configuration, the log information will be output to this directory
}
}
1.5 Return value description
| Returns value |
Description |
| The instance pointer |
Succeed |
| NULL |
Failed, at which point the parameters should be checked |
1.6 Sample code
long engine;
/* Create engine instance */
engine = AIEngine.aiengine_new("{\"appKey\": \""+ appKey +"\", \"secretKey\": \""+ secretKey +"\", \"provision\": \"assets/aiengine.provision\", \"cloud\":{\"enable\":1,\"server\": \"wss://cloud.chivox.com:443\"}}",null);
if (engine == 0)
{
System.out.println("create new engine failed");
return;
}
2.Make a request
2.1 Function prototype
- public static native int aiengine_start(long engine, String param, byte[] id, aiengine_callback callback, Object context);
2.2 Parameter description
| Argument |
Description |
| engine |
The pointer to the engine instance |
| param |
Start parameters, JSON format.
Includes three parts: app (app-related information), audio (audio format parameters), request (kernel-related parameters, see example 2.3 below for details).
- The audio parameters support the following two ways of audio data input
1.Microphone real-time audio data
This method is suitable for instant score scenes and supports only one audio format:
- wav: mono, 16Khz sample rate, 16bite sampling accuracy;
2.Recorded audio files
This method is suitable for scenes where non-instant score is not immediately available, the time it takes to evaluate audio is related to the length of audio, and the longer the audio, the longer the evaluation time. supports a variety of audio formats:- wav/pcm: mono, 16Khz sample rate, 16bite sampling accuracy; Note：1)To ensure scoring accuracy,wav audio files need to remove 44 bytes header; 2) pcm file audioType set to wav;
- mp3: mono, 16kHz sample rate, 16bite sampling accuracy;
- mp3: mono, 44.1kHz sample rate, 16bite sampling accuracy;
- ogg: mono, 16kHz sample rate, 16bite sampling accuracy;
|
| id |
requestId, an array of incoming empty characters before the call, in which the SDK records the unique request ID generated, corresponding to tokenId in the evaluation results |
| callback |
The callback function, the scoring results, and the exceptions in the score are all triggered into this callback function, see5.receive results |
| usrdata |
Callback parameters, passed in when aiengine_start, can be brought back as is in the callback function |
2.3 Evaluation request parameter description
| Name |
Type |
Option |
Description |
| param |
object |
required |
Review the content |
| - coreProvideType |
string |
required |
Set up "cloud" |
| - soundIntensityEnable |
int |
optional |
Whether to return the volume in real time, default 0, if set 1, the volume size through5.Receive the results of the onSoundIntensity interface callback,
the parameter is "sound_intensity", the value range 0 to 100; |
| - vad |
object |
optional |
Sound detection |
| - - vadEnable |
int |
optional |
The default is 0.
1 indicates that vaD functionality is enabled for this review.
0 indicates that vad is not enabled in this review. |
| - - refDuration |
int |
optional |
Sets the length of time the audio vad delay takes effect (in seconds),
which is to mask the VAD within seconds of the start of recording |
| - - speechLowSeek |
int |
optional |
Sensitivity, in 20ms, set N (default 15), indicates that the speech stops after 20 x N milliseconds are determined to be the end |
| - app |
object |
optional |
App-related information |
| - - userId |
string |
optional |
End-user identification.
It is recommended to fill in the user Id according to the user account number,
so as to facilitate troubleshooting. |
| - audio |
object |
required |
Audio information |
| - - audioType |
string |
required |
Audio encoding format |
| - - channel |
int |
required |
The number of audio channels |
| - - sampleBytes |
int |
required |
The number of audio samples |
| - - sampleRate |
int |
required |
Audio sample rate |
| - request |
object |
required |
Evaluation requests, different kernel request parameters are different, please refer to English Kernel Doc, Chinese Kernel Doc |
2.4 Param example
{
"coreProvideType": "cloud", // Required, online evaluation needs to be configured as "cloud"
"soundIntensityEnable": 0 //Optional, default 0, that is, no volume value is returned. If set to 1, the volume value is returned.
//The value is returned by callback, the parameter is "sound_intensity", the value range is 0-100
"vad": { //Optional, Sound detection function
"vadEnable": 0, //Optional, default 0. Setting 1 means the VAD function is enabled for this evaluation. Setting 0 means that the vad function is not enabled for this evaluation.
"refDuration": 2, //Optional, set the duration of audio vad delay (unit: seconds), that is, block VAD within a few seconds of the first recording
"speechLowSeek": 50 //Optional, sensitivity, unit 20ms, set to N, it means that 20*N milliseconds after the stop of speaking is judged to be the end
},
"app": { // part1: application related information
"userId": "guest", // Optional, the user ID in the application
},
"audio": { // part2: audio format parameters
"audioType": "wav", // required, audio encoding format
"channel": 1, // required, currently only supports mono, only 1
"sampleBytes": 2, // required, the number of bytes per sample, support: 1 (single byte, 8 bits) and 2 (double byte, 16 bits)
"sampleRate": 16000 // required, the sampling rate must be consistent with the actual audio
},
"request": { // voice service parameters (**see the kernel documentation** for details)
"coreType": "en.sent.score",
"refText": " I want to know the past and present of Hong Kong.",
"accent": 1,
"rank": 100,
"attachAudioUrl": 1,
}
}
**note: The request node has different kernel request parameters, please refer to English Kernel Doc, Chinese Kernel Doc
2.4 Return value description
| Return value |
Description |
| 0 |
Succeed |
| -1 |
Failed, at which point the aiengine_stop should be called immediately to get the reason for the failure |
|
|
2.5 Code sample
String refText = "\"I want to know the past and present of Hong Kong.\"";
String coretype = "en.sent.score";
final String json = String.format("{\"coreProvideType\": \"cloud\", \"app\":{\"userId\":\"aidemo\"}, \"audio\":{\"audioType\":\"wav\",\"channel\":1,\"sampleBytes\":2,\"sampleRate\":16000},\"request\":{\"coreType\":\""+coretype+"\", \"refText\":" + refText + ",\"rank\":100,\"attachAudioUrl\":1}}");
final byte[] id = new byte[64];
rv = AIEngine.aiengine_start(engine, json, id, callback, this);
3. Send audio data
3.1 Function prototype
- public static native int aiengine_feed(long engine, byte[] data, int size);
3.2 Function
- Perform specified actions, such as passing in audio data to the engine (audio data must have removed header information)
3.3 Parameters
| Argument |
Description |
| Engine |
The pointer to the engine instance |
| Data |
Audio data |
| Size |
Data size, recommended 320-64000 bytes |
3.4 Return value description
| Return value |
Description |
| 0 |
Succeed |
| -1 |
Fail |
3.5 Sample code
FileInputStream fis = null;
String audipath = "/audio/sent/sent.wav";
/*The local audipath*/
String audioFilePath = System.getProperty("user.dir")+ audipath;
if (rv == 0)
{
try
{
fis = new FileInputStream(audioFilePath);
} catch (FileNotFoundException e1)
{
// TODO Auto-generated catch block
e1.printStackTrace();
}
try
{
while ((bytes = fis.read(buf, 0, 1024)) > 0)
{
/* feed audio data */
if ((rv = AIEngine.aiengine_feed(engine,buf, bytes)) != 0)
break;
}
System.out.println("end read file(feed)");
} catch (IOException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
}
4.Stop the request
4.1 Function prototype
- public static native int aiengine_stop(long engine);
4.2 Function
- To end the engine's current request, the evaluation results are returned in the callback callback function set in the aiengine_start interface where the request was made
4.3 Parameters
| Argument |
Description |
| Engine |
The pointer to the engine instance |
4.4 Return value
| Return value |
Description |
| 0 |
Succeed |
| -1 |
Fail |
4.5 Sample code
rv = AIEngine.aiengine_stop(engine);
5.Receive the results
5.1 Function prototype
- public interface aiengine_callback {public abstract int run(byte[] usrdata, byte[] id, int type, byte[] message, int size);}
5.2 Function
- Asynchronous callback interface. After the stop request interface is called, the scoring results are called back through that interface. Exceptions in the scoring process are also triggered into this callback function
Note: No UI operations, IO operations, complex calculations, and any other operations that may cause blocking or waiting should be submitted to other threads for completion if necessary
5.3 Parameters
| Argument |
Description |
| usrdata |
Callback parameters, call aiengine_start when the incoming usrdata parameter is brought back as is |
| id |
RequestId, which corresponds to the unique identity of the request generated after the call aiengine_start |
| type |
The engine returns the message type, which is currently supported:- AIENGINE_MESSAGE_TYPE_JSON,
- AIENGINE_MESSAGE_TYPE_BIN (ONLY WHEN USING THE SPEECH SYNTHESIS KERNEL)
|
| message |
The message data returned by the engine |
| size |
The size of the message |
5.4 Sample code
private static AIEngine.aiengine_callback callback = new AIEngine.aiengine_callback() {
public int run(byte[] id, int type, byte[] data, int size) {
String recordId = new String(id, Charset.forName("UTF-8")).trim(); // must trim the end '\0'
System.out.println("in aiengine_callback...");
System.out.println("recordId: " + recordId);
if (type == AIENGINE_MESSAGE_TYPE_JSON) {
System.out.println("result: " + new String(data, 0, size, Charset.forName("UTF-8")).trim()); // must trim the end '\0'
textArea.setText(new String(data, 0, size, Charset.forName("UTF-8")).trim());
}
return 0;
}
};
6.Cancel the request
6.1 Function prototype
- public static native int aiengine_cancel(long engine);
6.2 Function
- After this method is called, the current evaluation request is canceled
6.3 Parameters
| Argument |
Description |
| Engine |
The pointer to the engine instance |
6.4 Return value
| Return value |
Description |
| 0 |
Succeed |
| -1 |
Fail |
6.5 Sample code
rv = AIEngine.aiengine_cancel(engine);
7. Destroy the engine
7.1 Function prototype
- public static native int aiengine_delete(long engine);
7.2 Function
- Destroy the engine instance
7.3 Parameters
| Argument |
Description |
| Engine |
The pointer to the engine instance |
7.4 Return value description
| Return value |
Description |
| 0 |
Succeed |
| -1 |
Fail |
7.5 Call method description
- Destroy the engine, which is recommended to be called when exiting the app.
7.6 Code sample
public void onDestory(){
super.onDestory();
if (engine != 0){
AIEngine.aiengine_delete(engine)
engine = 0;
}
if (recorder != null){
recorder.stop();
recorder = null;
}
System.exit(0);
}
8. Other interfaces
Get the version number
Function prototype
- public static native int aiengine_opt(long engine, int opt, byte[] data, int size);
Return value description
| Return value |
Description |
| The size of the data |
Normal |
| -1 |
Mistake |
Sample code
byte[]version = new byte[512];
int ret = AIEngine.aiengine_opt( 0, AIEngine.AIENGINE_OPT_GET_VERSION, version, 512 );
String sdkversion = new String(version,0, ret);
An example of the data
{
"version": "aiengine-256-windows_x86_64-2.2.7-20191012101218"
}
