[010.2] Continuous Integration with Semaphore

Setting up continuous integration for an Elixir project using Semaphore CI.

Subscribe now

Continuous Integration with Semaphore [07.27.2017]

What's the point of writing tests if you aren't going to ensure they're always passing? I think it's important to set up a continuous integration service very early in a project. Today we're going to introduce Semaphore CI for Firestorm. Let's get started.

Project

We're starting with the dailydrip/firestorm repo tagged before this episode.

We'll make a new branch, for later:

git checkout -b feature/semaphore_test
git push origin feature/semaphore_test

Setting up Semaphore is not particularly difficult. First, we'll log into our semaphore account, where we're presented with a list of our projects. I've already connected my GitHub account - you would have to go through that step first.

Semaphore Projects List

Next, we'll click Add new project to add Firestorm.

Semaphore Add Project

Then we'll pick the branch to build - we want to build the semaphore_test branch:

Semaphore Choose Branch

Finally, we pick the owner of the project.

Semaphore Choose Owner

Now semaphore will analyze the project, identify it as an Elixir project, and ask us to confirm the build steps.

Semaphore Build Steps

We'll ensure the latest version of Elixir is being used and apply these build steps. If we look, we can see that it hangs because there's no rebar3 and it wants us to confirm that it should install it, which of course we can't do as this is not interactive.

Could not find "rebar3", which is needed to build dependency :idna
I can install a local copy which is just used by Mix

Let's modify our build steps to first install rebar3, by adding the following step:

mix local.rebar --force

Now if we run the build again, it will fail because it can't connect to the database. Semaphore gives us a couple of environment variables that are meant to help here. First, let's move them to environment variable names of our choosing in Semaphore itself, in the build steps:

export POSTGRES_USER=$DATABASE_POSTGRESQL_USERNAME
export POSTGRES_PASSWORD=$DATABASE_POSTGRESQL_PASSWORD

We'll also need to set some environment variables, in the appropriate section in Semaphore's admin:

export AWS_ACCESS_KEY_ID=xxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxxxx
export AWS_S3_BUCKET=test-firestormforum-org
export AWS_S3_REGION=us-west-2
mix test

Then we'll update firestorm's test configuration to support using these environment variables for the repo's database setup:

vim config/test.exs

# Configure your database
config :firestorm_web, FirestormWeb.Repo,
  adapter: Ecto.Adapters.Postgres,
  username: System.get_env("POSTGRES_USER") || "postgres",
  password: System.get_env("POSTGRES_PASSWORD") || "postgres",
  database: "firestorm_web_test",
  hostname: "localhost",
  pool: Ecto.Adapters.SQL.Sandbox

If we push this up, we'll see a new build triggered. This build will likely fail due to a race condition involving our Notifications worker. Let's modify the application to avoid starting our Notifications process in test mode:

vim lib/firestorm_web/application.ex

defmodule FirestormWeb.Application do
  # We need the whole module to have access to the supervisor spec functions
  import Supervisor.Spec

  # ...
  def start(_type, _args) do
    opts = [strategy: :one_for_one, name: FirestormWeb.Supervisor]
    Supervisor.start_link(children(Mix.env), opts)
  end

  defp default_children() do
    [
      # Start the Ecto repository
      supervisor(FirestormWeb.Repo, []),
      # Start the endpoint when the application starts
      supervisor(FirestormWeb.Web.Endpoint, [])
    ]
  end
  defp children(:test) do
    default_children()
  end
  defp children(_) do
    default_children() ++
    [
      # Start the notifications server
      worker(FirestormWeb.Notifications, [])
    ]
  end
end

We'd like to start it for the duration of the notifications test, however:

vim test/notifications_test.exs

defmodule FirestormWeb.NotificationsTest do
  # ...
  setup do
    {:ok, _} = FirestormWeb.Notifications.start_link()

    :ok
  end
  # ...
end

With that, our tests should pass. They still don't quite, on the CI server. This seems to happen because of an issue when we redirect. I believe this is due to a bug in the version of phantomjs that is being used on Semaphore at present. I think that when phantomjs redirects, it fails to send the appropriate headers to keep us in the right Ecto sandbox. For now, we'll skip the thread feature test's watch tests as those are what are triggering the redirects. I could be wrong about this, but it's all I've been able to come up with so far.

vim test/feature/threads_test.exs

# ...
  @tag :pending
  test "creating a new thread", %{session: session} do
# ...
  @tag :pending
  test "creating a new thread when unauthenticated", %{session: session} do
# ...
  @tag :pending
  test "watching a thread", %{session: session} do
# ...

And we'll configure ExUnit to skip pending tests:

vim test/test_helper.exs

ExUnit.configure(exclude: [pending: true])
ExUnit.start()
# ...

We'll push this up, and the tests should pass.

Summary

In today's episode, we saw how to set up continuous integration using Semaphore CI for the Firestorm project. Aside from a simple tweak to support the environment variables that we're provided by Semaphore, it was completely trivial. I hope you'll take the opportunity to set up CI for all of your projects, given it's so easy :)

If you'd like to sign up for Semaphore, they've offered a coupon code for DailyDrip users. You can sign up with coupon code DAILYDRIP30 for a 30% discount for three months. The coupon expires on August 31, 2017.

See you soon!

Details