登录

[Technical post]:Thunder AI Application Development Guide 0.3 - - IoT ecosystem - 45IOT
12-27-2018 19:46:54 Views 3,347

[Technical post]:Thunder AI Application Development Guide 0.3

1. Introduction

1.1 Purpose

This document is intended to help developers quickly develop their own AI applications on the 845 AI KIt.

1.2 Scope

The main content of this document includes the code description of the sample app that calls the AI algorithm, how to access different cameras on AI KIt, how to use non-standard AndroidApi peripherals (such as LED), other notes and restrictions.

1.3 Intended Audience

Developers who want to develop applications on the 845 AI KIt.

2. Sample App (AlgoDemo)

2.1 Development environment
  1. Build SDK version:26
  2. CMake version:3.6.4111459
  3. NDK version:14.1.3816874
2.2 Project Structure

2.2.1 Java Code

  1. app/src/main/java/com/thundersoft/algodemo/activity/
  • CloudActivity.java:activity for cloud
  • MainActivity.java:application launch activity
  • VamActivity.java:An activity that runs FaceSDK with Camera API1(android.hardware.Camera)
  • VamCam2Activity.java:An activity that runs FaceSDK with Camera API1(android.hardware.camera2)
  1. app/src/main/java/com/thundersoft/algodemo/activity/
  • EnrollDialog.java:Alert dialog for enroll a new face
  1. app/src/main/java/com/thundersoft/algodemo/audio/
  • PcmRecorder.java:PCM data recorder
  • PcmRecordListener.java:PCM data recorder callback
  1. app/src/main/java/com/thundersoft/algodemo/entity/
  • FaceInfo.java:Face information entity class
  • Person.java:Person information entity for database
  1. app/src/main/java/com/thundersoft/algodemo/manager/
  • FaceDbManager.java:Database operation
  1. app/src/main/java/com/thundersoft/algodemo/vam/entity/
  • FaceFrame.java:Entity class for drawing face frame
  • MapperParams.java:Preview size and rendering size mapping entity class
  • ViewSize.java:View size entity class
  1. app/src/main/java/com/thundersoft/algodemo/vam/ui/
  • DataDrawable.java:Face frame drawable class
  • FdFramesUI.java/FdFramesUI2.java:A class for drawing frame for face detection
  • FrFramesUI.java/FrFramesUI2.java:A class for drawing frame for face recognition
  1. app/src/main/java/com/thundersoft/algodemo/vam/utils/
  • MapperParamsUtils.java:View size mapping relationship tool
  • Utils.java:other tools
  1. app/src/main/java/com/thundersoft/algodemo/vam/
  • Algo.java:Utility class for algorithms
  1. app/src/main/java/com/thundersoft/algodemo/
  • AlgoNative.java:Java layer for JNI
  • App.java:Application class
  • WakeUpService.java:Voice wake up service

2.2.2 Native Code

  1. h file in app/src/main/cpp/libyuv/
  • Head file for libyuv
  1. Other .h file in app/src/main/cpp/
  • Head file for FaceSDK
  1. app/src/main/cpp/
  • native-lib.cpp:native layer for JNI

2.2.3Library

  1. app/libs
  • Kiwilibrary.aar:Voice wake up SDK
  • litepal-2.0.0-src.jar:a database framework
  1. app/src/main/jniLibs/arm64-v8a

2.2.4Key methods

  1. public native int init():
    Description: FaceSDK initialization method, called once before FaceSDK works
    Parameter: none
    Return: Operation result,For a detailed definition, see the definition RDC_STATE in app/src/main/cpp/core/prompt.h
  2. public native int startFaceThread():
    Description: Start the thread for face detection and face recognition in FaceSDK
    Parameter: none
    Return: Operation result,For a detailed definition, see the definition RDC_STATE in app/src/main/cpp/core/prompt.h
  3. public native int stopEveryThreads():
    Description: Stop the thread for face detection and face recognition in FaceSDK
    Parameter: none
    Return: Operation result,For a detailed definition, see the definition RDC_STATE in app/src/main/cpp/core/prompt.h
  4. public native void yuv2Bgr(ByteBuffer src_y, int src_stride_y, ByteBuffer src_u, int src_stride_u, ByteBuffer src_v, int src_stride_v,ByteBuffer des)(AlgoNative.java):
    Description: Convert YUV data to BGR data
    Parameter:
    src_y Y plane data
    src_stride_y Y plane stride
    src_u U plane data
    src_stride_u U plane stride
    src_v V plane data
    src_stride_y V plane stride
    des Bytebuffer for receiving converted bgr data
    Return: none
  5. public native int setNewFrameWithBgr (ByteBuffer frameData):
    Description: Pass a bytebuffer carrying BGR data to FaceSDK for algorithm processing
    Parameter: frameData BGR data
    Return: Operation result,For a detailed definition, see the definition RDC_STATE in app/src/main/cpp/core/prompt.h
  6. public native FaceInfo[] getBoxes (boolean isSmoothed):
    Description: Fetch latest boxes array asynchronously
    Parameter: isSmoothed If it's equal to "true", then box-shaking can be minimized by timespace filter. But there will lead to some latency.
    Return: An array of FaceInfos, return null when no face is detected, we can get the gender, age, expression the face from the FaceInfo entity, as well as the feature value of the face.
  7. public native float calcSimilarity (float[] feature1,float[] feature2):
    Description: Calculate the mix distance between two samples
    Parameter:
    feature1 face feature value
    feature2 face feature value
    Return: Similarity of two features, the range is 0 - 1

3. Access the Camera

3.1 Built-in Camera

We can access the built-in camera via Camera API1 or Camera API2, select the camera with id 0 to turn on the built-in camera:
1. Camera API1:
Camera.open(Camera.CameraInfo.CAMERA_FACING_BACK);
2. Camera API2:
CameraManager manager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
manager.openCamera(0, mStateCallback, mBackgroundHandler);

4. Use of peripherals

4.1 LED

4.1.1 Interface
In AI KIt, we control the LED lights by directly operating the file nodes.,the path of three different LED lights is as follows:
Camera LED : (/sys/class/leds/red);
WiFi LED: (/sys/class/leds/wifi_red;/sys/class/leds/wifi_green/;/sys/class/leds/wifi_blue)
Power LED: (/sys/class/leds/wifi_red;/sys/class/leds/wifi_green/;/sys/class/leds/wifi_blue)

4.1.2 Instructions

  1. Turn on the LED light (the value can not be 0, the max is 255)
    echo 255 >> brightness
  2. Turn off the LED
    echo 0 >> brightness
  3. Turn on breathing (the value can not be 0,The higher the value, the lower the frequency of breathing)
    echo 1 >> blink
  4. Turn off breathing (Led lights will be turned off at the same time)
    echo 0 >> blink

4.1.3 Sample Code
Taking the Wifi LED as an example,the following function is implemented in the Java layer of the App to control the LED color and the breathing effect switch:

 bool setLED(int color, bool state ) {
    // TODO Auto-generated constructor stub
    int i;
    char raw_path[40] = {0};
     switch (color){
         case 1://red
             sprintf(raw_path, "/sys/class/leds/wifi_red/blink");
             break;
         case 2://green
             sprintf(raw_path, "/sys/class/leds/wifi_green/blink");
             break;
         case 3://blue
             sprintf(raw_path, "/sys/class/leds/wifi_blue/blink");
             break;
     }

    printf("led %s will turn %d\n", raw_path, color);
    int fd_data = open(raw_path, O_WRONLY );
    if (fd_data <= 0)
    {
        // fail to open the file
        exit(0);
    }

     if(state){
         int res = write(fd_data, "1", 1);// turn on the led light
     } else{
         int res = write(fd_data, "0", 1);// turn off the led light
     }
    close(fd_data);
    return true;
}
Top