Subscribe now

GenServer and Supervisor [03.01.2017]

Today we're going to look at the basics of OTP: GenServer and Supervisor. Let's get started.

Project

We'll be building a Fridge service. This is an actor that represents a Fridge you can put items into and take items out of. We could implement it using Agent, but a GenServer provides more flexibility for behaviour that's about more than just state management.

Then we'll supervise the Fridge, so that if it dies a new one is created for us.

GenServer

GenServer stands for Generic Server. Think of it as a shell for building actors that run concurrently.

We'll start off with the dailydrip/otp_playground repo, tagged before this episode. I've already provided a few tests to show how our Fridge will work:

defmodule OtpPlayground.FridgeServerTest do
  use ExUnit.Case
  alias OtpPlayground.FridgeServer

  test "putting something into the fridge" do
    { :ok, fridge } = GenServer.start_link FridgeServer, [], []
    assert :ok == GenServer.call(fridge, {:store, :bacon})
  end

  test "removing something from the fridge" do
    { :ok, fridge } = GenServer.start_link FridgeServer, [], []
    GenServer.call(fridge, {:store, :bacon})
    assert {:ok, :bacon} == GenServer.call(fridge, {:take, :bacon})
  end

  test "taking something from the fridge that isn't in there" do
    { :ok, fridge } = GenServer.start_link FridgeServer, [], []
    assert :not_found == GenServer.call(fridge, {:take, :bacon})
  end
end

Basically, you can create a new fridge, store items in it, and take items from it. Let's start introducing GenServer.

vim lib/otp_playground/fridge_server.ex

To start a GenServer, you just make a module and use GenServer.

defmodule OtpPlayground.FridgeServer do
  use GenServer
end

We're using the GenServer.start_link function to start our server. This takes three arguments: the GenServer's module, the arguments to pass it on initialization, and some options that start_link supports.

init/1 is what's referred to as a callback. When we use GenServer we specify that our module will follow the GenServer Behaviour. This means that promise to implement various functions that the GenServer module expects to exist. By default, we also had some basic implementations provided for us. We will need to provide our own implementation of init/1. It should return a two-tuple, {:ok, state}, where state is the persistent state for the server. Our fridge will be based on a list, so let's implement that:

defmodule OtpPlayground.FridgeServer do
  use GenServer

  def init(_) do
    {:ok, []}
  end
end

Next, let's try to run the tests:

mix test

We get some errors that look like this:

 1) test putting something into the fridge (OtpPlayground.FridgeServerTest)
     test/fridge_server_test.exs:5
     ** (EXIT from #PID<0.142.0>) an exception was raised:
         ** (RuntimeError) attempted to call GenServer #PID<0.143.0> but no handle_call/3 clause was provided
             (otp_playground) lib/gen_server.ex:559: OtpPlayground.FridgeServer.handle_call/3
             (stdlib) gen_server.erl:615: :gen_server.try_handle_call/4
             (stdlib) gen_server.erl:647: :gen_server.handle_msg/5
             (stdlib) proc_lib.erl:247: :proc_lib.init_p_do_apply/3

Another part of the expected behaviour is that you implement this handle_call/3 function for any GenServer.call that you want to handle. Our test does this:

GenServer.call(fridge, {:store, :bacon})

This means we expect to handle a call for {:store, :bacon}. To do that, you implement handle_call. It takes three arguments - the call, the pid that called (which you usually ignore until you have a much more interesting use case), and the current state. Let's handle this call, inserting the item into our state.

defmodule OtpPlayground.FridgeServer do
  use GenServer

  def init(_) do
    {:ok, []}
  end

  def handle_call({:store, item}, _from, state) do
    {:reply, :ok, [item | state]}
  end
end

The return value for our handle_call means reply with `:ok` and update the state by adding the new item to the front of the list.

Now our first test, inserting an item, passes. Next, we want to take items out of the fridge. We can make one of the two tests pass pretty easily by cheating:

defmodule OtpPlayground.FridgeServer do
  # ...
  def handle_call({:take, item}, _from, state) do
    {:reply, {:ok, item}, state}
  end
end

Here we're just always returning the item, whether or not it's in the fridge, and we're not removing it from the state either. The other test fails, and it will force us to handle this case. Let's look for the item in the fridge, and if it exists we'll return it and remove it from our state:

defmodule OtpPlayground.FridgeServer do
  # ...
  def handle_call({:take, item}, _from, state) do
    case Enum.member?(state, item) do
      true ->
        { :reply, {:ok, item}, List.delete(state, item) }
      false ->
        { :reply, :not_found, state }
    end
  end
end

Now all three tests pass. So this is a straightforward implementation of our Fridge. However, our API for interacting with it is extremely weird. We're expecting users to use GenServer.call to interact with it, and that means that they have to know we implemented our fridge as a GenServer. What we've built so far is usually referred to as the Server API. We also want to provide a Client API, which is how we actually want people to interact with this module.

Let's start with our own start_link function. We'll update our tests:

defmodule OtpPlayground.FridgeServerTest do
  use ExUnit.Case
  alias OtpPlayground.FridgeServer

  test "putting something into the fridge" do
    { :ok, fridge } = FridgeServer.start_link
    assert :ok == GenServer.call(fridge, {:store, :bacon})
  end

  test "removing something from the fridge" do
    { :ok, fridge } = FridgeServer.start_link
    GenServer.call(fridge, {:store, :bacon})
    assert {:ok, :bacon} == GenServer.call(fridge, {:take, :bacon})
  end

  test "taking something from the fridge that isn't in there" do
    { :ok, fridge } = FridgeServer.start_link
    assert :not_found == GenServer.call(fridge, {:take, :bacon})
  end
end

Now we'll implement FridgeServer.start_link - we'll just do what we were doing in our tests previously, but we'll optionally accept options to pass as the third argument to GenServer.start_link. We'll use those later; for now we can ignore them.

defmodule OtpPlayground.FridgeServer do
  use GenServer

  ### Client API
  def start_link(options \\ []) do
    GenServer.start_link __MODULE__, [], options
  end

  ### Server API
  # ...
end

Then we'll implement store and take in the same fashion. Let's update our tests to reflect what we want our API to be:

defmodule OtpPlayground.FridgeServerTest do
  use ExUnit.Case
  alias OtpPlayground.FridgeServer

  test "putting something into the fridge" do
    { :ok, fridge } = FridgeServer.start_link
    assert :ok == FridgeServer.store(fridge, :bacon)
  end

  test "removing something from the fridge" do
    { :ok, fridge } = FridgeServer.start_link
    FridgeServer.store(fridge, :bacon)
    assert {:ok, :bacon} == FridgeServer.take(fridge, :bacon)
  end

  test "taking something from the fridge that isn't in there" do
    { :ok, fridge } = FridgeServer.start_link
    assert :not_found == FridgeServer.take(fridge, :bacon)
  end
end

We'll implement these as helpers for the code we had in this test previously:

defmodule OtpPlayground.FridgeServer do
  use GenServer

  ### Client API
  def start_link(options \\ []) do
    GenServer.start_link __MODULE__, [], options
  end

  def store(pid, item) do
    GenServer.call(pid, {:store, item})
  end

  def take(pid, item) do
    GenServer.call(pid, {:take, item})
  end

  ### Server API
  # ...
end

Now the tests still pass, but a user of our module doesn't need to know that it's implemented as a GenServer. They also just generally have a nicer way to interact with the FridgeServer.

Supervisor

So we can start a FridgeServer, and interact with it. However, it could go down: someone could kill the process directly, or it could crash due to a bug. What if you need a FridgeServer to be around for the rest of your app, or at startup? This is where Supervisors come in.

Supervisors have a list of children that they are responsible for. They can be run in various modes. In one_for_one mode, which we'll be using, they restart each child independently if it dies. The other modes are important as well but a bit outside of the scope of this primer.

I created this app with mix new otp_playground --sup, which means it is an Application with a supervision tree. If you look in the mix.exs, you can see that our application has an application module callback.

defmodule OtpPlayground.Mixfile do
  use Mix.Project

  def project do
    [app: :otp_playground,
     version: "0.1.0",
     elixir: "~> 1.4",
     build_embedded: Mix.env == :prod,
     start_permanent: Mix.env == :prod,
     deps: deps()]
  end

  # Configuration for the OTP application
  #
  # Type "mix help compile.app" for more information
  def application do
    # Specify extra applications you'll use from Erlang/Elixir
    [extra_applications: [:logger],
     mod: {OtpPlayground.Application, []}] # <--
  end

  # Dependencies can be Hex packages:
  #
  #   {:my_dep, "~> 0.3.0"}
  #
  # Or git/path repositories:
  #
  #   {:my_dep, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}
  #
  # Type "mix help deps" for more examples and options
  defp deps do
    []
  end
end

Because we have an application function, Elixir will start that Application module's supervision tree when we boot the app. Let's look at the default that was provided:

vim lib/otp_playground/application.ex
defmodule OtpPlayground.Application do
  # See http://elixir-lang.org/docs/stable/elixir/Application.html
  # for more information on OTP Applications
  @moduledoc false

  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    # Define workers and child supervisors to be supervised
    children = [
      # Starts a worker by calling: OtpPlayground.Worker.start_link(arg1, arg2, arg3)
      # worker(OtpPlayground.Worker, [arg1, arg2, arg3]),
    ]

    # See http://elixir-lang.org/docs/stable/elixir/Supervisor.html
    # for other strategies and supported options
    opts = [strategy: :one_for_one, name: OtpPlayground.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

You can see that we have a start function. We can also see how to define children, and that this will start a Supervisor with strategy one_for_one. All that means is that each child will be restarted independently if it dies for any reason. We can make our FridgeServer start under this supervision tree by adding it as a child:

defmodule OtpPlayground.Application do
  # ...
  def start(_type, _args) do
    # ...
    children = [
      worker(OtpPlayground.FridgeServer, [])
    ]
    # ...
  end
end

This works, but how do we interact with it? We need to know its process ID to talk to it. We'll pass in the GenServer options - remember our optional argument to start_link - when we start the worker. We'll use the :name option to give our GenServer a name:

    children = [
      worker(OtpPlayground.FridgeServer, [[name: Fridge]])
    ]

Now when we start the app with iex -S mix it will start our FridgeServer and give it the name Fridge. We can interact with it:

iex -S mix
iex(1)> OtpPlayground.FridgeServer.store(Fridge, :bacon)
:ok
iex(2)> OtpPlayground.FridgeServer.take(Fridge, :bacon)
{:ok, :bacon}
iex(3)> OtpPlayground.FridgeServer.take(Fridge, :bacon)
:not_found

We can find the process id for Fridge using Process.whereis(Fridge):

iex(4)> Process.whereis(Fridge)
#PID<0.120.0>
# NOTE: You will almost certainly see a different Process ID here

Now we can kill it with Process.exit/2:

iex(5)> Process.whereis(Fridge) |> Process.exit(:kill)
true

Since it's supervised, it will have been restarted for us basically immediately:

iex(6)> Process.whereis(Fridge)
#PID<0.137.0>

Note that it has a new Process ID. It will also have lost its state, as it will have been restarted from scratch according to the worker specification.

Summary

So that's a relatively basic introduction to both GenServer and Supervisor. These are the foundational parts of OTP applications. By structuring your application as a series of supervision trees, you can achieve ridiculously high uptime. The key is designing your system such that each successive piece can be restarted in the event of a failure. I hope you enjoyed it. See you soon!

Resources