python + uiautomator2 Chinese usage rules
1, Installation
1. Install uiautomator2
#Since uiautomator2 is still under development, you must add '– pre' to install the development version.
pip install --pre uiautomator2
#You can also install from source
git clone https://github.com/openatx/uiautomator2
pip install -e uiautomator2
If you need a screenshot, install pilot
pip install pillow
2. Install device daemon
Connect one or more mobile phones to the computer and ensure that adb has been added to the environment variable. Execute the following commands to automatically install the device side programs required by the Library: uiautomator server, ATX agent, openstf/minicap, openstf/minitouch
#Initialize all devices already connected to the computer
python -m uiautomator2 init
Sometimes init will make mistakes. Please refer to manual Init
Guide, installation prompt success.
3. Install weditor
PIP install - U editor # currently, the latest stable version is 0.1.0
Windows system can use the command to create a shortcut Python - M editor -- shortcut on the desktop
Start Python -m WebEditor from the command line, and the browser will open automatically. Enter the ip or serial number of the device and click Connect.
2, Use guide
1. Connect the equipment
There are three connection methods:
• via WiFi
If the device IP is 10.0.0.1, your computer is on the same network
import uiautomator2 as u2
d = u2.connect('10.0.0.1') # alias for u2.connect_wifi('10.0.0.1')
print(d.info)
• via USB
Assume that the device number is 123456f (query through cmd: adb devices)
import uiautomator2 as u2
d = u2.connect('123456f') # alias for u2.connect_usb('123456f')
print(d.info)
• via ADB WiFi
import uiautomator2 as u2
d = u2.connect_adb_wifi("10.0.0.1:5555")
The u2.connect() function is called without parameters, and uiautomator2 will retrieve from the environment variable ANDROID_DEVICE_IP or ANDROID_SERIAL gets the device IP. If this environment variable is empty, uiautomator returns connect_usb, you need to make sure that only one device is connected to the computer.
2. Command line usage
Where $device_ip represents the ip address of the device.
To specify the device, you need to pass in - serial, such as Python 3 - M uiautomator2 -- Serial bff1234. Subcommand is a subcommand (init, screenshot, etc.)
1.0.3 added: Python 3 -m uiautomator2 can be abbreviated as uiautomator2
init: Install the required programs for the device uiautomator2 init If you need specify device to init, pass --serial <serial> python3 -m uiautomator2 init --serial your-device-serial screenshot: screenshot $ python -m uiautomator2 screenshot screenshot.jpg uninstall: uninstall python -m uiautomator2 uninstall <package-name> # Uninstall a package python -m uiautomator2 uninstall <package-name-1> <package-name-2> # Uninstall multiple packages python -m uiautomator2 uninstall --all # Uninstall all install: install apk，apk adopt URL give (Temporarily unavailable) clear-cache: wipe cache (It is being abandoned, and there is no need to change the interface at present) app-stop-all: Stop all applications (temporarily unavailable) healthcheck: health examination (Temporarily unavailable)
3, API manual
1. Global setting
#Set the delay of 1.5 seconds between clicking again after each UI click
d.click_post_delay = 1.5 # Default no delay
Set default element wait timeout (seconds)
d.wait_timeout = 30.0 # The default is 20.0 seconds
2. Retrieve equipment information
d.info
The following are possible results:
{
u'displayRotation': 0,
u'displaySizeDpY': 640,
u'displaySizeDpX': 360,
u'currentPackageName': u'com.android.launcher',
u'productName': u'takju',
u'displayWidth': 720,
u'sdkInt': 18,
u'displayHeight': 1184,
u'naturalOrientation': True
}
3. Keyboard operation
On / off screen
d.screen_on() # Turn on screen d.screen_off() # Close screen
Get screen on / off status
d.info.get('screenOn') # Android > = 4.4 required
Press the hard / soft key
d.press("home") # Press the home key
d.press("back") # Normal way to press the back key
d.press(0x07, 0x02) # Press codes 0x07('0 ') and 0x02(META ALT)
Currently supported keys
o home o back o left o right o up o down o center o menu o search o enter o delete ( or del) o recent (recent apps) o volume_up o volume_down o volume_mute o camera o power
You can use the Android KeyEvnet
All key code definitions found on.
#Unlock screen d.unlock()
#1. Start activity: com.github.uiautomator.ACTION_IDENTIFY
2. Press "home"
4. Gesture interaction of equipment
Click on the screen
d.click(x, y)
Long press the screen
d.long_click(x, y) d.long_click(x, y, 0.5) # long click 0.5s (default)
slide
d.swipe(sx, sy, ex, ey) d.swipe(sx, sy, ex, ey, 0.5) # swipe for 0.5s(default)
drag
d.drag(sx, sy, ex, ey) d.drag(sx, sy, ex, ey, 0.5) # swipe for 0.5s(default) be careful: click, swipe, drag Support percentage position. example: d.long_click(0.5, 0.5) #Press and hold the center of the screen
5. Screen operation
Retrieve / set direction
Available directions are:
natural or n
left or l
right or r
upsidedown or u (not settable)
#The retrieval direction can be "natural" or "left" or "right" or "upsidedown"
orientation = d.orientation
#Warning: failed the test on my TT-M1
#Set orientation and freeze rotation
d.set_orientation("n") # Or "natural"
d.set_orientation("l") # Or "left"
d.set_orientation("r") # Or "right"
d.set_orientation("n") # or "natural"
Freeze / thaw rotation
#Freeze rotation d.freeze_rotation() #Unfreeze rotation d.freeze_rotation(False)
screenshot
#Take a screenshot and save it to the local file "home.jpg"
d.screenshot("home.jpg")
#To obtain PIL.Image format, you need to install pilot first
image = d.screenshot()
image.save("home.jpg") # Or home.png
#To obtain the opencv format, you need to install numpy and cv2 first
import cv2
image = d.screenshot(format='opencv')
cv2.imwrite('home.jpg', image)
Dump window hierarchy
#Get the contents of the dump (unicode) xml = d.dump_hierarchy()
Turn on notifications or quick settings
d.open_notification() d.open_quick_settings()
6. Push and extract files
Push file to device
＃Push to a folder
d.push("foo.txt", "/sdcard/")
#Push and rename
d.push("foo.txt", "/sdcard/bar.txt")
#Push fileobj
with open("foo.txt", 'rb') as f:
d.push(f, "/sdcard/")
#Push and modify file mode
d.push("foo.sh", "/data/local/tmp/", mode=0o755)
Extract file from device
d.pull("/sdcard/tmp.txt", "tmp.txt")
#If there is no file in the device, FileNotFoundErr will be raised
d.pull("/sdcard/some-file-not-exists.txt", "tmp.txt")
7.APP management
Including APP installation, startup and shutdown
APP installation
Currently, only url installation is supported.
d.app_install(' HTTP: //some-domain.com/some.apk ')
APP start
d.app_start("com.example.hello_world") # Start with package name
APP stop
#Execute forced stop
d.app_stop("com.example.hello_world")
#Perform application cleanup
d.app_clear('com.example.hello_world')
Stop the operation of all apps
＃ Stop all d.app_stop_all() ＃Stop Division com.examples.demo All applications except d.app_stop_all(excludes=['com.examples.demo'])
8. Selector Selecor
Selectors are used to identify specific ui objects in the current window
#Select an element whose text is' Clock 'and className is' android.widget.TextView' d(text='Clock', className='android.widget.TextView')
The selector supports the following parameters. See UiSelecor java doc for details
.
• text, textContains, textMatches, textStartsWith
• className, classNameMatches
• description, descriptionContains, descriptionMatches, descriptionStartsWith
• checkable, checked, clickable, longClickable
• scrollable, enabled,focusable, focused, selected
• packageName, packageNameMatches
• resourceId, resourceIdMatches
• index, instance
Get children, grandchildren and sibling UI objects
Children and grandchildren
#Get child or grandchild (recursive get) d(className="android.widget.ListView").child(text="Bluetooth")
Same level
#Get sibling or child of sibling d(text="Google").sibling(className="android.widget.ImageView")
Use text or description or instance to obtain children and grandchildren level objects
#Obtain the matching class name "android.widget.LinearLayout" in the children and grandchildren, and the text contains "Bluetooth"
d(className="android.widget.ListView", resourceId="android:id/list")\ .child_by_text("Bluetooth", className="android.widget.LinearLayout")
#Allow page scrolling search for children and grandchildren
d(className="android.widget.ListView", resourceId="android:id/list") \
.child_by_text(
"Bluetooth",
allow_scroll_search=True,
className="android.widget.LinearLayout"
)
o child_by_description is used to obtain the object containing the specified description in the children and grandchildren, and the rest are and child_ by_ Textsame
o child_by_instance is used to get a specific instance in the child and grandchild objects visible on the screen
For more information, see the following link:
o UiScrollable
, getChildByDescription, getChildByText, getChildByInstance
o UiCollection
, getChildByDescription, getChildByText, getChildByInstance
The above method supports chained calls, for example, for the following hierarchy
<node index="0" text="" resource-id="android:id/list" class="android.widget.ListView" ...>
<node index="0" text="WIRELESS & NETW or KS" resource-id="" class="android.widget.TextView" .../>
<node index="1" text="" resource-id="" class="android.widget.LinearLayout" ...>
<node index="1" text="" resource-id="" class="android.widget.RelativeLayout" ...>
<node index="0" text="Wi Fi" resource-id="android:id/title" class="android.widget.TextView" .../>
</node>
<node index="2" text="ON" resource-id="com.android.settings:id/switchWidget" class="android.widget.Switch" .../>
</node>
...
</node>
System Setting
We want to turn on Wi Fi by clicking the switch to the right of "Wi Fi". Since several switches have almost the same properties, we cannot use switches like
d(className = "android.widget.Switch") to select objects. You can use the following code to select objects.
d(className="android.widget.ListView", resourceId="android:id/list")\
.child_by_text("Wi Fi", className="android.widget.LinearLayout")\
.child(className="android.widget.Switch")\
.click()
relative position
We can use the relative position method to obtain the objects in the current view: left, right, top, bottom
o d(A).left(B) indicates that B to the left of A is selected
o d(A).right(B) means to select B on the right side of A
o d(A).up(B) means to select B above A
o d(A).down(B) indicates that B under A is selected
Therefore, for the "Wi Fi" switch, we can write the following code:
#Select "switch" to the right of "Wi Fi" d(text="Wi Fi").right(className="android.widget.Switch").click()
multiple instances
Sometimes there are multiple objects with the same characteristics (such as text) on the screen, you will have to use the "instance" attribute in the selector, as follows:
d(text="Add new", instance=0) # Gets the first element object with the text "Add new"
UI Automator also provides a list - like method to handle similar element objects
#Gets the total number of elements with the text "Add new" on the current screen d(text="Add new").count
#The len function has the same function as the count attribute
len(d(text="Add new"))
#Get element instance through index
d(text="Add new")[0] d(text="Add new")[1]
#iteration
for view in d(text="Add new"):
view.info # ...
Note: when using selectors such as lists, you must ensure that the screen remains unchanged, otherwise ui not found error may appear
Get the status and information of the ui object
Check whether the specified ui object exists
d(text="Settings").exists # True if present, False otherwise d.exists(text="Settings") # Alias of the above property
Retrieves information about the specified ui object
d(text="Settings").info
The following are possible results:
{ u'contentDescription': u'',
u'checked': False,
u'scrollable': False,
u'text': u'Settings',
u'packageName': u'com.android.launcher',
u'selected': False,
u'enabled': True,
u'bounds': {u'top': 385,
u'right': 360,
u'bottom': 585,
u'left': 200},
u'className': u'android.widget.TextView',
u'focused': False,
u'focusable': True,
u'clickable': True,
u'chileCount': 0,
u'longClickable': True,
u'visibleBounds': {u'top': 385,
u'right': 360,
u'bottom': 585,
u'left': 200},
u'checkable': False
}
Set / clear text for editable fields
d(text="Settings").clear_text() # Clear text
d(text="Settings").set_text("My text...") # Set text
click the specified ui object
Click specify ui object
#Click the center of the UI object d(text="Settings").click() #The maximum time to wait (element display) is 10 seconds, and click (default). If the timeout is not displayed, an error will be reported d(text="Settings").click(timeout=10) #click alias, short name of keyboard operation d(text="Settings").tap() #Unequal element show d(text="Settings").tap_nowait() Long press specified ui object #Long press and hold to specify ui object d(text="Settings").long_click()
Specify ui object gesture action
Drag an object to a point or another object
#Note: drag cannot be set before Android 4.3 #Drag ui object to point (x, y) d(text="Settings").drag_to(x, y, duration=0.5) #Drag a ui object to another ui object (Center) d(text="Settings").drag_to(text="Clock", duration=0.25)
Two point gesture
d(text="Settings").gesture((sx1, sy1), (sx2, sy2), (ex1, ey1), (ex2, ey2)) # s start point, e end point
Specify the ui object two-point gesture
Two gestures are supported:
o In, from edge to center
o Out, from center to edge
#Note: scaling cannot be set before Android 4.3 #From edge to center. d(text="Settings").pinch_in(percent=100, steps=10) #From center to edge d(text="Settings").pinch_out()
Wait for the specified ui object to appear or disappear
#Wait for the ui object to appear d(text="Settings").wait(timeout=3.0) # return bool #Wait for the ui object to disappear d(text="Settings").wait_gone(timeout=1.0)
The default timeout is 20s. See "1. Global setting" for details
fling (scrollable) ui objects
Flying is generally used for turning pages horizontally or vertically. Possible properties:
o horiz or vert
o forward or backward or toBeginning or toEnd
#Flying defaults to vertical forward d(scrollable=True).fling() #fling horizontally forward d(scrollable=True).fling.horiz.forward() #Flying vertical backward d(scrollable=True).fling.vert.backward() vertical fling To the beginning d(scrollable=True).fling.h or iz.toBeginning(max_swipes=1000) vertical fling To the end d(scrollable=True).fling.toEnd()
scroll the (scrollable) ui object
Possible attributes:
o horiz or vert
o forward or backward or toBeginning or toEnd, or to
#scroll defaults to vertical forward d(scrollable=True).scroll(steps=10) #scroll horizontal forward d(scrollable=True).scroll.h or iz.forward(steps=100) #scroll vertical backward d(scrollable=True).scroll.vert.backward() #Horizontal scroll to start d(scrollable=True).scroll.h or iz.toBeginning(steps=100, max_swipes=1000) #Vertical scroll to end d(scrollable=True).scroll.toEnd() #scroll vertically forward until the specified ui object appears d(scrollable=True).scroll.to(text="Security")
9. Trigger (Watcher)
You can register triggers to perform ui object actions that the selector does not match.
Register a named trigger (Watcher)
When the selector cannot find a match, uiautomator will run all registered triggers
Click the target when the conditions match
d.watcher("AUTO_FC_WHEN_ANR").when(text="ANR").when(text="Wait") \
.click(text="Force Close")
#d.watcher(name) # Create and name a trigger
#d.when(condition) # Trigger conditions
#d.click(target) # click the target object
Press key when matching parts
d.watcher("AUTO_FC_WHEN_ANR").when(text="ANR").when(text="Wait") \
.press("back", "home")
#d.watcher(name) # Create and name a trigger
#d.when(condition) # Trigger conditions
#d .press(<keyname>, ..., <keyname>.() # Press the key in turn
Check whether the Watcher with the specified name has been triggered
A Watcher has been triggered, which means that all trigger conditions match and the Watcher program has been run
d.watcher("watcher_name").triggered # true if triggered, otherwise false
Delete a named trigger
#Delete trigger
d.watcher("watcher_name").remove()
List all triggers
d.watchers # Returns a list of registered triggers
Check whether any Watcher has been triggered
d.watchers.triggered # true if any trigger is triggered, otherwise false
Reset all triggered triggers
#Reset all triggered triggers, and then d.watchers.triggered returns False d.watchers.reset()
Delete trigger
#Delete all registered triggers
d.watchers.remove()
#Delete the specified trigger, and the effect is the same as d.watcher("watcher_name").remove()
d.watchers.remove("watcher_name")
Force all triggers to run
#Force all registered triggers to run d.watchers.run()
In addition, there are many articles not written in this article. It is recommended to directly see the source code init.py
10. Input method
It is usually used for input without knowing the control. The first step is to switch the input method, and then send the adb broadcast command. The specific use method is as follows
d.set_fastinput_ime(True) # Switch to FastInputIME input method
d.send_keys("Hello 123 abcEFG") # adb broadcast input
d.clear_text() # Clear all contents of the input box (Android uiautomator. APK version > = 1.0.7 is required)
d.set_fastinput_ime(False) # Switch to normal input method
11.Toast message
Display Toast message
d.toast.show("Hello world")
d.toast.show("Hello world", 1.0) # show for 1.0s, default 1.0s
Get Toast
#[Args] #5.0: maximum wait timeout. Default 10.0 #10.0: cache time. If toast has occurred in the last 10 seconds, the cached toast is returned. Default 10.0 (may change in the future) #"default message": if you don't get toast in the end, it will be returned. Default no d.toast.get_message(5.0, 10.0, "default message") #General usage assert "Short message" in d.toast.get_message(5.0, default="") #Clear cache toast d.toast.reset() #Now d.toast.get_message(0) is None
12.XPath positioning
Example: content of a node
<android.widget.TextView index="2" text="05:19" resource-id="com.netease.cloudmusic:id/qf" package="com.netease.cloudmusic" content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false" password="false" selected="false" visible-to-user="true" bounds="[957,1602][1020,1636]" />
xpath positioning and usage
Some attribute names need to be modified
description -> content-desc
resourceId -> resource-id
Common usage
#wait exists 10s
d.xpath("//android.widget.TextView").wait(10.0)
#find and click
d.xpath("//*[@ content desc = 'share'] ". click()
#check exists
if d.xpath("//android.widget.TextView[contains(@text, 'Se')]").exists:
print("exists")
#get all text-view text, attrib and center point
for elem in d.xpath("//android.widget.TextView").all():
print("Text:", elem.text)
# Dictionary eg:
# {'index': '1', 'text': '999+', 'resource-id': 'com.netease.cloudmusic:id/qb', 'package': 'com.netease.cloudmusic', 'content-desc': '', 'checkable': 'false', 'checked': 'false', 'clickable': 'false', 'enabled': 'true', 'focusable': 'false', 'focused': 'false','scrollable': 'false', 'long-clickable': 'false', 'password': 'false', 'selected': 'false', 'visible-to-user': 'true', 'bounds': '[661,1444][718,1478]'}
print("Attrib:", elem.attrib)
# Coordinate eg: (100, 200)
print("Position:", elem.center())
For other common usage of XPath, see: https://github.com/openatx/uiautomator2/blob/master/uiautomator2/ext/xpath/README.md
13. Jiugong grid unlock
Having said so much, go straight to the code.
import uiautomator2 as u2 u = u2.connect() # Connect your phone to your PC u.swipe_points([(0.235, 0.456), (0.503, 0.449), (0.509, 0.601), (0.777, 0.603), (0.771, 0.763), (0.222, 0.75)], 0.2)
Where (0.235, 0.456) represents x (23.5%) and Y (45.6%). Absolute coordinates can also be used here.
The last 0.2 represents the time of each slide.
14. Frequently asked questions
Many things that are not written in this place are put here, Common Issues
Stop UiAutomator
Stop UiAutomator daemon service
https://github.com/openatx/uiautomator2/wiki/Common-issues
Due to the existence of ATX agent, uautomator will be guarded all the time. If it quits, it will be restarted. However, uautomator is overbearing. Once it is running, the auxiliary functions on the mobile phone and the uiautomatorviewer on the computer will not work unless the uiautomator of the framework itself is turned off. The following two closing methods are described
Method 1:
Directly open the uiautomator app (after init succeeds, it will be installed). Click to close UIAutomator
Method 2:
d.service("uiautomator").stop()
#d.service("uiautomator").start() # start-up
#d.service("uiautomator").running() # Is it running
AccessibilityService method for coexistence of ATX and Maxim
14. Difference between uiautomator and Uiautomator2
https://www.cnblogs.com/insist8089/p/6898181.html
4, Frequently asked questions
1. Prompt 502 error
Try connecting your phone to your PC, and then run the following command
adb shell am instrument -w -r -e debug false -e class com.github.Uiautomator.stub.Stub
com.github.Uiautomator.test/android.support.test.runner.AndroidJUnitRunner
If it works, add a line of code before starting the test
d.healthcheck()
If an error is reported, an apk may be missing and not installed. Use the following command to reinitialize
python -m Uiautomator2 init --reinstall
1. This article is from the open source project uiautomator2: https://github.com/openatx/uiautomator2
2. Later update Python+uiautomator2 mobile phone UI automation test practice