How to add Flutter integration test in a CI with GitHub Action ?

Introduction

During the development of a project, we need something to make integration testing. And more importantly, use a CI to automatically run the tests. We will see in this article how to make our first integration test and use Github Action to perform the test in a workflow.

Tools we will need:

• Flutter => install here
• An account GitHub

Simple integration test

Before we deep dive into the CI workflow we need a simple integration test, to do so we will create a simple application with a test inside it:

flutter create integration_testing_with_ci

Then we create the test:

cd integration_testing_with_ci
mkdir integration_test
mkdir test_driver

First, we will create the integration_test.dart inside test_driver/ :

import 'dart:io';
import 'package:integration_test/integration_test_driver_extended.dart';

Future<void> main() async {
  try {
    await integrationDriver(
      onScreenshot: (String screenshotName, List<int> screenshotBytes) async {
        final File image = await File('screenshots/$screenshotName.png').create(recursive: true);
        image.writeAsBytesSync(screenshotBytes);
        return true;
      },
    );
  } catch (e) {
    print('Error occured: $e');
  }
}

We can see here we add some code to handle the screenshot folder.

Next, we need the test itself  increment_success_test.dart:

import 'package:flutter_test/flutter_test.dart';
import 'package:integration_test/integration_test.dart';

import 'package:integration_testing_with_ci/main.dart' as app;

void main() {
  final binding = IntegrationTestWidgetsFlutterBinding();
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();

  group('increment', () {
    testWidgets('success', (WidgetTester tester) async {
      app.main();
      await binding.convertFlutterSurfaceToImage();
      await tester.pumpAndSettle();

      await binding.takeScreenshot('test-screenshot');

      // Verify the counter starts at 0.
      expect(find.text('0'), findsOneWidget);

      // Finds the floating action button to tap on.
      final Finder fab = find.byTooltip('Increment');

      // Emulate a tap on the floating action button.
      await tester.tap(fab);

      // Trigger a frame.
      await tester.pumpAndSettle();

      // Verify the counter increments by 1.
      expect(find.text('1'), findsOneWidget);
    });
  });
}

This is a simple test you can find in the official documentation.

You can now run the integration test locally with the command:

flutter drive --driver=test_driver/integration_test.dart --target=integration_test/increment_success_test.dart

You should find a screenshot inside screenshots/ folder.

Workflow Github Action

We will use GitHub to make our workflow. First, init your repository with git init , then create your .github/workflows/ folders with a file named test-integration.yaml.

The most important action we will use is reactivecircus/android-emulator-runner@v2, we can find the documentation here. It will run an emulator android and execute some script on it.

We will now create the workflow:

on:
  push:
    tags-ignore:
      - '**'
    branches:
      - '**'
name: Flutter integration test
jobs:
  drive_android:
    runs-on: macos-latest
    strategy:
      matrix:
        api-level: [29]
        target: [playstore]
    steps:
      - uses: actions/checkout@v2
      - uses: subosito/flutter-action@v1
        with:
          flutter-version: '2.8.1'
          channel: 'stable'

      # Run integration test
      - name: Run Flutter Driver tests
        uses: reactivecircus/android-emulator-runner@v2
        with:
          target: ${{ matrix.target }}
          api-level: ${{ matrix.api-level }}
          arch: x86_64
          profile: Nexus 6
          script: flutter drive --driver=test_driver/integration_test.dart --target=integration_test/increment_success_test.dart

      # Update screenshot to artifact
      - name: Upload screenshots
        if: always()
        uses: actions/upload-artifact@v1
        with:
          name: screenshot
          path: ${{ matrix.path }}screenshots/

The most important in this workflow is the job "Run Flutter Driver tests", the action will start an emulator for us then execute the script provided and here we use the same command as before in local.

And finally, we upload the screenshots in artifact so we can check it later.

We just have to push the file into the repository and the workflow will be triggered:

And if you click on the workflow you should see something like this:

And inside the jobs we see some logs:

And after some minutes you should see:

And if you click on "Summary" you should see your artifact (screenshots):

We can see that our test pass successfully in the CI, and use the emulator provided by reactivecircus .

Thanks to the matrix strategy we can also run the workflow for multi api levels for example this is a workflow with 21, 23, and 29 for target default and google_apis:

We just have to add in the yaml:

jobs:
  drive_android:
    runs-on: macos-latest
    strategy:
      matrix:
        api-level: [21, 23, 29]
        target: [default, google_apis]

You can find all the possible configurations inside the documentation:

About prices

We use GitHub action and for the price, it's free for repositories public and we got 2,000 minutes/month for private repo.

All the prices can be found here:

Conclusion

With this workflow, we can now run integration testings directly in GitHub. And see the result/screenshots in an artifact.

Like always if you need to check the code take a look at my repository: