Getting started with Appium for mobile testing

Mobile applications are increasingly becoming complex as they provide a wide range of functionalities, catering to diverse use cases across finance, health, entertainment, and other industries. Application developers need to ensure compatibility across a wide range of devices, operating systems, and screen sizes. It is challenging to ensure a bug-free user experience, as manually testing all the features across devices would require a lot of time and effort.

To overcome this challenge, it is critical to automate end-to-end (E2E) testing so that teams can release new features with confidence and speed. Appium is a popular open-source mobile testing framework that lets you automate UI testing for native Android and iOS applications. Appium is a cross-platform framework that supports test clients in multiple programming languages and easily integrates with CI/CD tools like CircleCI.

In this article, you will learn about how Appium works, its architecture, its benefits, and how to set it up to test an Android application. The article demonstrates how to use WebDriver.io, which is a NodeJS Appium client, to write tests for an Android application. Instead of using a NodeJS client, you could also use Appium Java client, but this tutorial uses NodeJS as its lightweight, easier to maintain and beginner friendly. You will also learn how to integrate your tests with BrowserStack cloud devices and CircleCI CI/CD platform for continuous testing.

Prerequisites

Before you get started, you will need the following:

• Android Studio installed and the Android development environment set up. The setup should include adding a mobile device emulator to Android Studio.
• A CircleCI account
• The latest version of Appium inspector installed. You can find the release binary under GitHub releases.
• NodeJS installed
• A BrowserStack account. You can sign up for a free trial.

Overview of Appium

Appium uses the closed-box testing approach, which lets you test the mobile app without modifying the source code. The benefit of this approach is that you can test the exact same app bundle that gets distributed to the end users. The following sections discuss its benefits and architecture in more detail.

Benefits of using Appium

Appium is a cross-platform testing tool that supports multiple programming languages and allows UI testing of Android, iOS, and hybrid mobile applications. Some of its benefits are:

• Cross-platform testing: You can test Android, iOS, and hybrid mobile applications using the same test scripts, allowing developers to write once and test everywhere. It significantly reduces the testing time and effort as developers don’t need to write and maintain platform-specific test scripts. The range of locator strategies supported by Appium makes it compatible with different UI rendering frameworks and languages including Jetpack Compose UIs, React-Native, Flutter, and Swift.
• Support for multiple programming languages: It lets you write tests in the programming language of your choice as it provides Appium clients for Java, Python, Ruby, .Net, and NodeJS. Additionally, any W3C WebDriver spec compatible client can be integrated with Appium, allowing developers the flexibility to extend Appium to a currently non-supported programming language.
• No code modification: Appium supports black-box testing, allowing you to test the Android APK or iOS IPA, without any modifications. This allows faster testing as the artifact built for distribution can be tested directly before publishing.
• CI/CD integration: It can be seamlessly integrated with your existing CI/CD tools, allowing you to continuously test your application with every change.

Appium’s architecture

Appium follows a client-server architecture where the Appium Client sends test commands to the Appium Server, which then interacts with the device under test using the platform-specific automation framework. Refer to the architecture diagram to view the different components involved.

Let us go over the different components and understand how they work together:

• Appium Client: It refers to the client framework that is used to write the test script in Java, Python, or Javascript and send the automation commands to the server.
• Appium Server: The server receives the automation commands from the client, processes the request, and forwards it to the platform-specific automation framework.
• Automation Frameworks: The platform-specific automation framework receives commands from the Appium server and executes the action on the test device. For example, UiAutomator2 is a driver that can execute test commands on Android devices and XCUiTest can be used to execute commands on iOS devices.
• Mobile devices: It refers to the device under test that receives the test command from the automation framework and executes the requested action.

Setting up the Android app

In this section, you’ll set up a basic Android application that you can later use for Appium E2E testing. Clone this Github repository and switch to the `starter branch.

git clone https://github.com/CIRCLECI-GWP/CircleCIAppiumDemo
cd CircleCIAppiumDemo
git checkout starter

Open the CircleCIAppiumDemo/android project using Android Studio. Let’s go over the code setup before running the application:

• The MainActivity.kt contains a login form with username and password text fields. After entering a dummy username and password, when the user clicks the Login button, a login success message is shown, indicating successful login. If either of these fields are left empty, an error message is displayed, indicating login failure.
• The service/UserService.kt contains a dummy login method that checks the input username and password and returns the appropriate login message. Notice that the service doesn’t call any API to perform an actual login, as we only need a dummy service to emulate the login.

To run the app, click the Run app button in Android Studio.

The first build might take a couple of minutes to complete, and once the app is built, Android Studio will automatically launch the application on an emulator.

You can test the app’s functionality by entering a random username and password and verifying the functionality.

Finally, run the assembleDebug command to build and create the debuggable version of the APK, as you will need it for E2E testing.

# execute at project root
cd android
./gradlew assembleDebug

Once the command completes execution, verify that the APK is present at this path:

# relative path under project root
./android/app/build/outputs/apk/debug/app-debug.apk

Setting up the E2E test directory

Now that the Android application is set up, let us focus on adding E2E tests using Webdriver.io. Webdriver.io is a NodeJS testing framework that lets you write E2E tests for both web and mobile applications. In this tutorial, we will use it for Appium-based E2E testing. Start by creating an e2e directory in the project’s root.

Use this command:

# execute in the project's root
mkdir e2e

Adding package.json

You need to create a package.json file in the NodeJS E2E test directory to define the runner scripts, dependencies, etc. Add a file named package.json to the e2e directory and add this to it:

{
    "name": "circleci-android-browserstack-demo",
    "engines": {
        "node": "^16.13.0 || >=18.0.0"
    },
    "type": "module",
    "scripts": {
        "android:local": "wdio run wdio.android.local.conf.js",
        "android:browserstack": "wdio run wdio.android.browserstack.conf.js"
    },
    "devDependencies": {
        "@wdio/appium-service": "^9.7.1",
        "@wdio/browserstack-service": "^9.7.1",
        "@wdio/cli": "^9.7.1",
        "@wdio/globals": "^9.7.1",
        "@wdio/local-runner": "^9.7.1",
        "@wdio/mocha-framework": "^9.6.4",
        "@wdio/spec-reporter": "^9.6.3",
        "appium": "^2.15.0",
        "appium-uiautomator2-driver": "^3.10.0",
        "eslint-plugin-wdio": "^9.6.0"
    }
}

The contents in the package.json file:

• Define scripts for running E2E tests locally and on BrowserStack. Notice that the scripts reference the wdio.android.local.conf.js and wdio.android.browserstack.conf.js files, which will be defined later in this section.
• Add the required dependencies for Appium, Webdriver.io, and BrowserStack.

After adding the file, install the dependencies needed by executing:

# navigate to the e2e directory
cd e2e
npm install

Define local Webdriver.io config

You will need the ability to run the tests locally on simulators to verify the E2E tests, before running them on BrowserStack devices. To facilitate this, define a e2e/wdio.android.local.conf.js file with the following contents:

export const config = {
    // Base config
    specs: ["tests/specs/**/app*.spec.js"],
    logLevel: 'info',
    bail: 0,
    waitforTimeout: 45000,
    connectionRetryTimeout: 120000,
    connectionRetryCount: 3,
    framework: 'mocha',
    reporters: ['spec'],
    mochaOpts: {
        ui: 'bdd',
        timeout: 3 * 60 * 1000, // 3 min
    },

    services: [
        [
            'appium',
            {
                args: {
                    relaxedSecurity: true,
                    log: './logs/appium.log',
                },
            },
        ],
    ],

    before: async () => {
        if (driver.isAndroid) {
            await driver.updateSettings({
                waitForSelectorTimeout: 3 * 1000,
            });
        }
    },

    capabilities: [
        {
            platformName: "Android",
            "wdio:maxInstances": 1,
            "appium:deviceName": "Pixel_XL_API_33",
            "appium:appPackage": "com.example.circleciappiumdemo",
            "appium:platformVersion": "13.0",
            "appium:orientation": "PORTRAIT",
            "appium:automationName": "UiAutomator2",
            "appium:app": "../android/app/build/outputs/apk/debug/app-debug.apk",
            "appium:appWaitActivity": "com.example.circleciappiumdemo.MainActivity",
            "appium:newCommandTimeout": 240,
        }
    ],
};

Here’s a breakdown of the config file:

• It defines the test spec path that determines the E2E tests that will be executed when this config is executed.
• It sets the test framework as mocha and sets appium as the service that will be used to run the tests.
• It also defines the Android capabilities that tells Appium which device, platform version, and APK to use for testing. Notice that the appium:app refers to the path of the APK that you built in the previous section.
• If the appium:deviceName and appium:platformVersion don’t match your emulator’s name and OS version, update the values. While adding the device name, you need to replace the whitespaces with an underscore (_). For example, if the emulator name shows as “Pixel XL API 33” in Android Studio, you need to add Pixel_XL_API_33 as appium:deviceName.

Note, that the tutorial uses static capabilities, but for convenience, you can move the dynamic configs to a dotenv file and configure it using the CI environment variables.

Define the test spec file

Finally, add an E2E test that enters a test username and password in the input fields and verifies that the login message contains the input username. First create a a e2e/tests/specs/ directory by executing:

# execute this command in the project's root
mkdir -p e2e/tests/specs/

Once the directory is created, create a e2e/tests/specs/app.login.spec.js file and add the following contents to it:

describe("Android Login Test", () => {
    it("should login with valid credentials", async () => {
        const usernameField = await driver.$('//android.widget.EditText[@resource-id="com.example.circleciappiumdemo:id/user_name_field"]');
        const passwordField = await driver.$('//android.widget.EditText[@resource-id="com.example.circleciappiumdemo:id/user_password_field"]');
        const loginButton = await driver.$('//android.widget.Button[@resource-id="com.example.circleciappiumdemo:id/login_button"]');
        const displayUserName = await driver.$('//android.widget.TextView[@resource-id="com.example.circleciappiumdemo:id/display_user_name"]');

        await usernameField.setValue("testUser");
        await passwordField.setValue("password123");
        await loginButton.click();
        await driver.pause(2000);
        const displayedText = await displayUserName.getText();
        expect(displayedText).toContain("testUser");
    });
});

Here’s a breakdown of the code snippet’s key parts:

• The Webdriver.io test spec file uses Mocha.js testing framework to define the test. The describe block lets you group multi tests in a single block and the it block defines a single test case. Also, notice that the test uses await in multiple places to wait for the fulfillment of the Promise before continuing execution.

• Notice that the test uses Xpath selectors to find the elements on the screens. For example, the username input field has the XPath //android.widget.EditText[@resource-id="com.example.circleciappiumdemo:id/user_name_field"]. You can use Appium inspector to find the XPath of the elements on the screen.

• Also, Webdriver commands to perform input and click actions. For example, the setValue command is used to input the testUser text in the usernameField. Similarly, the click API is used to click the loginButton.
• Finally, the expect API is used to verify that the displayed text contains testUser.

Run tests locally

Before running the tests locally, make sure that an Android emulator is running in Android Studio. Its name and platform version should match the capabilities defined in wdio.android.local.conf.js. Execute the following command to run the tests locally:

# navigate to the e2e directory
cd e2e
npm run android:local

Once you execute the command, the APK will be installed on the emulator, and it will execute the test spec defined in app.login.spec.js. Confirm that the test passes successfully.

Configuring the tests to run in BrowserStack Device Cloud

BrowserStack is a cloud-based web and mobile test automation platform that provides cloud-based real mobile devices that can be used for native application testing. For Appium, it provides the App Automate product. App Automate can be used for testing native applications using Appium.

Now that you have run the E2E tests on a local simulator, let us add a configuration file to run it on BrowserStack cloud devices. In the e2e directory’s root, create a file called wdio.android.browserstack.conf.js and add the following contents to it:

export const config = {
    specs: ["tests/specs/**/app*.spec.js"],

    capabilities: [
        {
            'appium:app': process.env.BROWSERSTACK_APP_ID || 'BROWSERSTACK_APP_ID',
            'appium:deviceName': 'Samsung Galaxy S22 Ultra',
            'appium:platformVersion': '12.0',
            'platformName': 'Android',
            'bstack:options': {
                debug: true,
                projectName: 'wdio-test-project',
                buildName: 'android',
                sessionName: 'wdio-test'
            }
        }
    ],

    user: process.env.BROWSERSTACK_USERNAME || 'BROWSERSTACK_USER',
    key: process.env.BROWSERSTACK_ACCESS_KEY || 'BROWSERSTACK_ACCESS_KEY',
    services: ['browserstack'],

    logLevel: 'info',
    bail: 0,
    waitforTimeout: 45000,
    connectionRetryTimeout: 120000,
    connectionRetryCount: 3,

    framework: 'mocha',
    reporters: ['spec'],

    mochaOpts: {
        ui: 'bdd',
        timeout: 3 * 60 * 1000, // 3min
    },
};

The above configuration file is very similar to the local config except for a few differences:

• It has the user and key properties that set the username and access key used for connecting to your BrowserStack account. Refer to the linked guide to retrieve your username and password, and keep it handy.
• The appium:app capability points to a BrowserStack app path. You can either use the REST API to upload the app or upload the APK through the BrowserStack dashboard. Use either of the options to upload the APK and keep the app URL handy.

Now that the config file is defined, set the following environment variables in your terminal before triggering the tests:

export BROWSERSTACK_USERNAME='<YOUR_BROWSERSTACK_USERNAME>'
export BROWSERSTACK_ACCESS_KEY='<YOUR_BROWSERSTACK_ACCESS_KEY>'
export BROWSERSTACK_APP_ID='<BROWSERSTACK_APP_URL>'

Substitute the actual values and execute the above commands to set the environment variables. Note, that if you are using a Windows device, you need to replace export with set. For example,

set BROWSERSTACK_USERNAME=<YOUR_BROWSERSTACK_USERNAME>
set BROWSERSTACK_ACCESS_KEY=<YOUR_BROWSERSTACK_ACCESS_KEY>
set BROWSERSTACK_APP_ID=<BROWSERSTACK_APP_URL>

Next, execute the following command to run the tests on BrowserStack:

npm run android:browserstack

Once the command completes execution, head over to the BrowserStack dashboard and click the test session details to view test details.

Automate E2E tests using CircleCI

Now that you have run E2E tests and triggered the BrowserStack tests locally, let us automate E2E testing on BrowserStack using CircleCI. The CircleCI workflow will build the Android application, upload the APK to BrowserStack, and run the E2E tests on BrowserStack cloud devices.

Add the CircleCI configuration script

First, create a .circleci directory in the project’s root by executing:

mkdir .circleci

Once the directory is created, create a .circleci/config.yaml script containing the configuration file for the CI pipeline. Add this code snippet to it.

version: 2.1

executors:
  android-executor:
    docker:
      - image: cimg/android:2025.02
    working_directory: ~/project

  e2e-executor:
    docker:
      - image: cimg/node:20.10
    working_directory: ~/project/e2e

jobs:
  build:
    executor: android-executor
    steps:
      - checkout
      - run:
          name: Grant execute permission to Gradle wrapper
          command: chmod +x android/gradlew
      - run:
          name: Build Android App
          command: |
            cd android
            ./gradlew assembleDebug
      - store_artifacts:
          path: android/app/build/outputs/apk/debug/app-debug.apk
          destination: app-debug.apk
      - run:
          name: Upload APK to BrowserStack
          command: |
            APP_URL=$(curl -u "$BROWSERSTACK_USERNAME:$BROWSERSTACK_ACCESS_KEY" \
                 -X POST "https://api-cloud.browserstack.com/app-automate/upload" \
                 -F "file=@android/app/build/outputs/apk/debug/app-debug.apk" | jq -r '.app_url')
            echo "APP_URL=$APP_URL" >> /tmp/browserstack_app_url.env
      - persist_to_workspace:
          root: /tmp
          paths:
            - browserstack_app_url.env

  e2e-test:
    executor: e2e-executor
    steps:
      - checkout
      - attach_workspace:
          at: /tmp
      - run:
          name: Load APP_URL into Environment
          command: echo "export BROWSERSTACK_APP_ID=$(cat /tmp/browserstack_app_url.env | cut -d '=' -f2)" >> $BASH_ENV
      - run:
          name: Print BROWSERSTACK_APP_ID
          command: echo "BROWSERSTACK_APP_ID=$BROWSERSTACK_APP_ID"
      - run:
          name: Install E2E Dependencies
          command: |
            cd e2e
            npm install
      - run:
          name: Run E2E Tests on BrowserStack
          command: |
            cd e2e
            npm run android.browserstack.app

workflows:
  version: 2
  build_and_test:
    jobs:
      - build
      - e2e-test:
          requires:
            - build

Take a moment to review the CircleCI configuration:

• The config defines the build job that uses the cimg/android:2025.02 image to build the Android APK.
• The build job also uploads the APK to BrowserStack using the REST API and persists the app_url to the workspace.
• It also defines the e2e-test job that uses the cimg/node:20.10 image to run E2E tests.
• It installs the NPM dependencies and runs the npm run android.browserstack.app script to execute tests on a BrowserStack cloud device.

Now that the configuration file has been properly set up, create a repository for the project on GitHub and push all the code to it. Review Pushing a project to GitHub for instructions.

Setting up the project on CircleCI

Next, log in to your CircleCI account.

On the CircleCI dashboard, click the Projects tab, search for the GitHub repo name, and click Set Up Project for your project.

You will be prompted to add a new configuration file manually or use an existing one. Since you have already pushed the required configuration file to the codebase, select the Fastest option. Enter the name of the branch hosting your configuration file. Click Set Up Project to continue.

Completing the setup will trigger the pipeline but it will fail since the environment variables are not set.

Set environment variables

On the project page, click Project settings

From Project Settings, go to the Environment variables tab. On the screen that appears, click Add environment variable button to add environment variables.

Go ahead and add the following environment variables.

• BROWSERSTACK_USERNAME to the BrowserStack username. You can retrieve the username by following this guide.
• BROWSERSTACK_ACCESS_KEY to the BrowserStack access key. Similar to the username you can retrieve the access key by following this guide.

Once you add the environment variables, it should show the key values on the dashboard.

Testing the pipeline

Now that the environment variables are configured, trigger the pipeline again by clicking the Trigger Pipeline button. This time the build should succeed.

You can click the build pipeline to view details for Android and upload to BrowserStack steps.

Similarly, click the e2e-test pipeline to view details for the E2E testing steps.

Once the pipeline finishes execution, head over to the BrowserStack App Automate dashboard to verify that the tests passed.

You can click the test session to view details of test execution.

Conclusion

In this tutorial, you learned how to automatically build the Android app and run Appium E2E tests on BrowserStack using CircleCI. Appium provides an open-source platform for writing and running UI tests for native and hybrid mobile applications. BrowserStack is a cloud-based platform that provides access to hundreds of real Android and iOS devices that can facilitate UI testing across device models, screen sizes, and geographical locations.

With CircleCI, you can automate the build and testing pipeline for continuous integration. The pipeline can be used to build the Android app, upload the APK to BrowserStack, and run E2E tests on real cloud-based Android devices.

You can check out the complete source code used in this tutorial on GitHub.