GitHub Apps: How to do acceptance testing

For testing GitHub App integrations, we've created a private repository called github-app-testing in the @coderly account.

The github-app-testing repo contains stub issues, pull requests, and comments that allow us to do acceptance testing. The repo is private solely because we assert certain things in tests that could break if the repository were made public.

The testing environment

In your .env.example you should see a number of GITHUB_TEST_ environment variables. If you're a volunteer, you won't be able to run these acceptance tests locally or in CircleCI within your fork. We unfortunately cannot expose the PEM file or secret keys.

These acceptance tests will run on Circle by running tests that include the tag @acceptance: true.

If you're not a volunteer and do have access to our credentials, you can run these acceptance tests locally with mix test.acceptance.

To set up your testing environment, you'll need to:

• Acquire the credentials from the "Code Corps Testing" GitHub App
• Base64 encode the PEM file
• Place all the credentials in your .env

These credentials will also need to be available on Circle for the tests to run properly.

Test failures

If you're seeing acceptance tests fail, likely reasons include:

• the data has changed in the stub github-app-testing repository
• you're missing credentials

HTTP requests

Keep in mind that these acceptance requests do make HTTP requests. These requests should not be mocked or stubbed because it's testing full acceptance of our integration with the GitHub API.