Android Accessibility Service

Keywords: Android xml encoding

  1. When the service is not turned on, quickly jump to the interface to turn on the service.
if (!OpenAccessibilitySettingHelper.isAccessibilitySettingsOn(this,
            AccessibilitySampleService.class.getName())){// Judge whether the service is on
          OpenAccessibilitySettingHelper.jumpToSettingPage(this);// Jump to open page
        }else {
          Toast.makeText(this, "Service turned on", Toast.LENGTH_SHORT).show();
        }

The specific implementation of the methods used:

/**
 * Open accessibility service help class
 * Created by mazaiting on 2017/8/18.
 */
public class OpenAccessibilitySettingHelper {

  /**
   * Jump to accessibility settings page
   * @param context device context
   */
  public static void jumpToSettingPage(Context context){
    Intent intent = new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS);
    intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK);
    context.startActivity(intent);
  }

  /**
   * Judge whether there is auxiliary function permission
   * @return true Already opened
   *          false Not opened
   */
  public static boolean isAccessibilitySettingsOn(Context context,String className){
    if (context == null){
      return false;
    }
    ActivityManager activityManager = (ActivityManager) context.getSystemService(Context.ACTIVITY_SERVICE);
    List<ActivityManager.RunningServiceInfo> runningServices =
        activityManager.getRunningServices(100);// Get the list of running services
    if (runningServices.size()<0){
      return false;
    }
    for (int i=0;i<runningServices.size();i++){
      ComponentName service = runningServices.get(i).service;
      if (service.getClassName().equals(className)){
        return true;
      }
    }
    return false;
  }
}

2. For simulation click, create the Activity of simulation click as AccessibilityNormalSampleActivity, and configure AccessibilityNormalSampleActivity and AccessibilitySampleService in the same process in AndroidManifest.xml. If they are not in the same process, the AccessibilityService and AccessibilityEvent obtained are empty.

android:process=":BackgroundService"

The configuration file is:

<!-- Register accessibility services -->
    <service
        android:name=".service.AccessibilitySampleService"
        android:enabled="true"
        android:exported="true"
        android:label="@string/accessibility_tip"
        android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE"
        android:process=":BackgroundService">

      <!-- android:label="@string/accessibility_tip" Text displayed in settings -->
      <intent-filter>
        <action android:name="android.accessibilityservice.AccessibilityService" />
      </intent-filter>
      <!-- adopt xml The auxiliary functions can be configured in the onServiceConnected Medium dynamic configuration -->
      <meta-data
          android:name="android.accessibilityservice"
          android:resource="@xml/accessibility_config" />
    </service>

    <activity android:name=".ui.AccessibilityNormalSampleActivity"
        android:process=":BackgroundService"></activity>

AccessibilityNormalSampleActivity interface layout:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/activity_accessibility_main"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:paddingBottom="@dimen/activity_vertical_margin"
    android:paddingLeft="@dimen/activity_horizontal_margin"
    android:paddingRight="@dimen/activity_horizontal_margin"
    android:paddingTop="@dimen/activity_vertical_margin"
    android:orientation="vertical">

  <CheckBox
      android:id="@+id/normal_sample_checkbox"
      android:layout_width="wrap_content"
      android:layout_height="wrap_content"
      android:text="Check box switch"/>

  <RadioButton
      android:id="@+id/normal_sample_radiobutton"
      android:layout_width="wrap_content"
      android:layout_height="wrap_content"
      android:layout_marginTop="10dp"
      android:text="radio button"/>

  <ToggleButton
      android:id="@+id/normal_sample_togglebutton"
      android:layout_width="wrap_content"
      android:layout_height="wrap_content"
      android:layout_marginTop="10dp"/>

  <Button
      android:id="@+id/normal_sample_back"
      android:layout_marginTop="20dp"
      android:text="Exit this page"
      android:layout_width="match_parent"
      android:layout_height="wrap_content" />
</LinearLayout>

3. Create a singleton class to control the accessibility operator.

/**
 * Control accessibility services
 * Created by mazaiting on 2017/8/18.
 */
public class AccessibilityOperator {
  private static final String TAG = "AccessibilityOperator";
  private static AccessibilityOperator mInstance;
  private AccessibilityOperator(){}
  public static AccessibilityOperator getInstance() {
    if (mInstance == null){
      synchronized (AccessibilityOperator.class){
        if (mInstance == null){
          mInstance = new AccessibilityOperator();
        }
      }
    }
    return mInstance;
  }
}
  1. Create an interface to simulate clicks
    Create a Handler in the Activity to be clicked to execute the delay message, create the AccessibilityOperator object, and obtain the singleton object in the onCreate method.
  private Handler mHandler = new Handler(Looper.getMainLooper());
  private AccessibilityOperator accessibilityOperator;

  @Override protected void onCreate(Bundle savedInstanceState) {
      .....// Omit layout fill
      accessibilityOperator = AccessibilityOperator.getInstance();
    }

And click in the onResume method. Only the method in AccessibilityOperator is called here, so paste the code directly:

@Override protected void onResume() {
    super.onResume();
    // Perform delayed tasks
    clickText();
  }

  /**
   * Click by text
   */
  private void clickText() {
    clickTextItem("check box",1);
    clickTextItem("radio button",2);
    clickTextItem("Close",3);
    clickTextItem("Exit this page",4);
  }

/**
   * Text single delayed Click
   * @param text Text content
   * @param num Delay multiple
   */
  private void clickTextItem(final String text,int num) {
    mHandler.postDelayed(new Runnable() {
      @Override public void run() {
        final boolean isSuccess = accessibilityOperator.clickText(text);
        runOnUiThread(new Runnable() {
          @Override public void run() {
            popToast(isSuccess, text);
          }
        });
      }
    },2000*num);
  }

  /**
   * Pop up toast
   * @param isSuccess
   * @param msg
   */
  private void popToast(boolean isSuccess, String msg) {
    if (isSuccess) {
      Toast.makeText(this, msg + "Click success", Toast.LENGTH_SHORT).show();
    } else {
      Toast.makeText(this, msg + "Clicking failed", Toast.LENGTH_SHORT).show();
    }
  }
  1. In the accessibilityOperator.clickText(text) text, the system will first call the onAccessibilityEvent method in the AccessibilitySampleService, so create an updateEvent method in the AccessibilityOperator to assign values to the AccessibilityService service and AccessibilityEvent events.
  private AccessibilityService mAccessibilityService;
  private AccessibilityEvent mAccessibilityEvent;
  /**
   * Update events
   * @param service
   * @param event
   */
  public void updateEvent(AccessibilityService service, AccessibilityEvent event) {
    if (mAccessibilityService == null && service != null){
      mAccessibilityService = service;
    }
    if (event != null){
      mAccessibilityEvent = event;
    }
  }
  1. After the AccessibilityService and AccessibilityEvent are assigned, they can be used normally. The complete content code of the clickText method:
  /**
   * Search all eligible nodes according to Text, fuzzy search method
   * @param text
   * @return
   */
  public boolean clickText(String text) {
    AccessibilityNodeInfo nodeInfo = getRootNodeInfo();
    if (nodeInfo!=null){
      List<AccessibilityNodeInfo> nodeInfos =
          nodeInfo.findAccessibilityNodeInfosByText(text);
      return performClick(nodeInfos);
    }
    return false;
  }

/**
   * Get root node
   * @return
   */
  private AccessibilityNodeInfo getRootNodeInfo() {
    Log.e(TAG, "getRootNodeInfo: ");
    AccessibilityEvent curEvent = mAccessibilityEvent;
    AccessibilityNodeInfo nodeInfo = null;
    if (Build.VERSION.SDK_INT >= 16){
      if (mAccessibilityService!=null){
        // Get form root
        nodeInfo = mAccessibilityService.getRootInActiveWindow();
      }
    }else {
      nodeInfo = curEvent.getSource();
    }
    return nodeInfo;
  }

  /**
   * Simulated Click
   * @param nodeInfos
   * @return true Success; false failure.
   */
  private boolean performClick(List<AccessibilityNodeInfo> nodeInfos) {
    if (nodeInfos!=null && !nodeInfos.isEmpty()){// Judge whether it is not empty
      AccessibilityNodeInfo nodeInfo;
      for (int i=0;i<nodeInfos.size();i++){
        nodeInfo = nodeInfos.get(i);// Get the View to click
        // Make a simulation Click
        if (nodeInfo.isEnabled()){// If you can click
          return nodeInfo.performAction(AccessibilityNodeInfo.ACTION_CLICK);
        }
      }
    }
    return false;
  }

AccessibilityNodeInfo returned by getRootNodeInfo() can be traversed to query its child nodes

    AccessibilityNodeInfo nodeInfo = getRootNodeInfo();
    if (nodeInfo!=null){
      for (int i=0;i<nodeInfo.getChildCount();i++){
        AccessibilityNodeInfo child = nodeInfo.getChild(i);
        Log.e(TAG, "clickText: "+child.toString());
      }
    }

Posted by thewabbit1 on Wed, 11 Dec 2019 12:11:26 -0800