[262] Phoenix Is An Interface, Not Your Application

All of this has happened before. All of this will happen again.

Subscribe now

Phoenix Is An Interface, Not Your Application [08.23.2016]

When I ran a consultancy doing a lot of Ruby work, I would teach our developers not to fall into the trap of thinking that Rails was their application. I only felt qualified to teach them this because I spent a few years in this pit myself.

This is why I felt so silly when I found myself doing this in Phoenix the other day. So let's talk about it.

Project

Recently, I was building a pretty basic Phoenix Channels application. I had an Elm application that managed some state, and I wanted to make it collaborative. I was going to put the state on the server and then share it with the clients and serialize access to it. Nothing fancy, no CRDTs...just basic stuff. The elm application was managing a song, which consisted of some tracks, and each track had some slots in it.

The simplest mistake

So of course I did this:

mix phoenix.new some_backend
cd some_backend
mix phoenix.gen.model Song songs
mix phoenix.gen.model Track tracks song_id:references:songs slots:map

OK, let's stop here for a moment and think about what just happened. I wanted to synchronize some state. I knew I wanted websockets so of course I was going to use Phoenix Channels because they're awesome. And then I lost my mind.

Why do I need a database?

Did I make that decision thoughtfully, after deliberation, considering the actual needs of my application?

I can kill the suspense for you. I did not. I was on autopilot, just derping along.

So if this isn't what I should do, what should I do?

Keep calm and use processes

Let's think back to the title. Phoenix is not your application. What is my application? It's actually something to deal with songs. We aren't going to build anything to collaborate on songs because that would ruin part of my ElixirConf talk next week, but we can talk about the basic problem at hand. We'll make an application that lets users collaborate on some super basic state: they can increment and decrement a counter together.

First, let's make an umbrella project:

mix new shared_counters --umbrella
cd shared_counters/apps

We'll make an application that manages our counters.

mix new count_chocula --sup

OK, we know we want a counter for some shared state. That means we want a GenServer at least, so let's write some tests for it.

mkdir test/count_chocula
vim test/count_chocula/server_test.exs
defmodule CountChocula.ServerTest do
  use ExUnit.Case
  alias CountChocula.Server

  test "starting the server" do
    assert {:ok, _pid} = Server.start_link()
  end
end

Right, so we want to be able to start our GenServer. It won't work, of course, but we run the test anyway so we can properly TDD.

Now we'll make the server:

mkdir lib/count_chocula
vim lib/count_chocula/server.ex
defmodule CountChocula.Server do
  use GenServer

  def start_link do
  end
end

And run the tests...

Right, we need to return {:ok, pid}. We'll actually stub out a server rather than just making this test pass:

defmodule CountChocula.Server do
  use GenServer

  ### Public API
  def start_link do
    GenServer.start_link(__MODULE__, [])
  end

  ### Server API
  def init(_) do
    {:ok, 0}
  end
end

Now everything's gravy. We should be able to ask what the state is, so we know what our counter shows:

  test "getting the count" do
    assert {:ok, pid} = Server.start_link()
    assert 0 = Server.get_count(pid)
  end

This fails, so we'll implement it:

defmodule CountChocula.Server do
  use GenServer

  ### Public API
  def start_link do
    GenServer.start_link(__MODULE__, [])
  end

  def get_count(pid) do
    GenServer.call(pid, :get_count)
  end

  ### Server API
  def init(_) do
    {:ok, 0}
  end

  def handle_call(:get_count, _from, state) do
    {:reply, state, state}
  end
end

We want to be able to increment:

  test "incrementing" do
    assert {:ok, pid} = Server.start_link()
    :ok = Server.increment(pid)
    assert 1 = Server.get_count(pid)
  end

We can't increment yet, so we'll make that pass:

defmodule CountChocula.Server do
  use GenServer

  ### Public API
  # ...
  def increment(pid) do
    GenServer.cast(pid, :increment)
  end

  ### Server API
  # ...
  def handle_cast(:increment, state) do
    {:noreply, state+1}
  end
end

Decrementing seems like it would be useful, but let's be honest...we don't need that, this is just so we have something to interact with. We'll stop here with the GenServer for now. Next, we'd like to have a supervisor that manages these for us, and we want to be able to give them names so we can look them up by URL. We'll add a test for that:

vim test/count_chocula/supervisor_test.exs
defmodule CountChocula.SupervisorTest do
  use ExUnit.Case, async: true

  describe "creating new counter" do
    test "creating a nonexistent counter" do
      assert {:ok, _pid} = CountChocula.Supervisor.start_counter(:some_id)
    end
  end
end

It fails, so let's make the supervisor. We'll assume we can start a child with an id, and we'll modify the worker to make that true.

defmodule CountChocula.Supervisor do
  use Supervisor
  @name CountChoculaSupervisor

  def start_link do
    Supervisor.start_link(__MODULE__, [], name: @name)
  end

  def start_counter(id) do
    Supervisor.start_child(@name, [id])
  end

  def init(_) do
    children = [
      worker(CountChocula.Server, [], restart: :transient)
    ]

    supervise(children, strategy: :simple_one_for_one)
  end
end
defmodule CountChocula.Server do
  use GenServer

  ### Public API
  def start_link(id) do
    GenServer.start_link(__MODULE__, [], name: {:global, id})
  end
  # ...
end

Run the test, and it will fail because our supervisor was never started. We'll add it to the application:

vim lib/count_chocula.ex
defmodule CountChocula do
  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      supervisor(CountChocula.Supervisor, [])
    ]

    opts = [strategy: :one_for_one]
    Supervisor.start_link(children, opts)
  end
end

OK, so let's run the test. It passes. What happens if we try to start a couple with the same name though? First, we'll just make a new test that starts one with the same name our previous test used:

defmodule CountChocula.SupervisorTest do
  use ExUnit.Case, async: true

  describe "creating new counter" do
    test "creating a nonexistent counter" do
      assert {:ok, _pid} = CountChocula.Supervisor.start_counter(:some_id)
    end

    test "creating a counter when one already exists by that name" do
      assert {:ok, _pid} = CountChocula.Supervisor.start_counter(:some_id)
      # More stuff here later
    end
  end
end

Run the tests, and they'll fail because now we're trying to register two processes with the same name. This is because our server that the previous test started is still running.

  1) test creating new counter creating a nonexistent counter (CountChocula.SupervisorTest)
     test/count_chocula/supervisor_test.exs:5
     match (=) failed
     code: {:ok, _pid} = CountChocula.Supervisor.start_counter(:some_id)
     rhs:  {:error, {:already_started, #PID<0.122.0>}}
     stacktrace:
       test/count_chocula/supervisor_test.exs:6: (test)

We want this to be what we test, but we don't want it to be incidental just because a previous test registered a process by that name. Let's introduce a unique ID so we can be sure we're getting new processes when we want them:

vim mix.exs
  defp deps do
    [
      { :uuid, "~> 1.1" }
    ]
  end
mix deps.get
vim test/test_helper.exs
ExUnit.start

defmodule TestHelper do
  def new_id() do
    UUID.uuid4(:hex)
  end
end

Now we'll use this from our test when we want new IDs:

vim test/count_chocula/supervisor_test.exs
defmodule CountChocula.SupervisorTest do
  use ExUnit.Case, async: true
  import TestHelper

  describe "creating new counter" do
    test "creating a nonexistent counter" do
      assert {:ok, _pid} = CountChocula.Supervisor.start_counter(new_id())
    end

    test "creating a counter when one already exists by that name" do
      the_id = new_id()
      assert {:ok, pid} = CountChocula.Supervisor.start_counter(the_id)
      assert {:ok, ^pid} = CountChocula.Supervisor.start_counter(the_id)
      # More stuff here later
    end
  end
end

So now, what happens when we intentionally start one with the same ID?

  1) test creating new counter creating a counter when one already exists by that name (CountChocula.SupervisorTest)
     test/count_chocula/supervisor_test.exs:10
     match (=) failed
     code: {:ok, _pid} = CountChocula.Supervisor.start_counter(the_id)
     rhs:  {:error, {:already_started, #PID<0.169.0>}}
     stacktrace:
       test/count_chocula/supervisor_test.exs:13: (test)

The same error as before. This time, let's catch it. We can tweak our API to pretend it started just fine in this case, and just return the existing process id:

defmodule CountChocula.Server do
  use GenServer

  ### Public API
  def start_link(id) do
    case GenServer.start_link(__MODULE__, [], name: {:global, id}) do
      {:ok, pid} -> {:ok, pid}
      {:error, {:already_started, pid}} -> {:ok, pid}
    end
  end
  # ...
end

OK, now if we run this test it passes. If we look at our Server test though, it's been broken since we introduced the id as an argument. We'll pull in our TestHelper there and get a new id for each test as well:

defmodule CountChocula.ServerTest do
  use ExUnit.Case
  alias CountChocula.Server
  import TestHelper

  test "starting the server" do
    assert {:ok, _pid} = Server.start_link(new_id())
  end

  test "getting the count" do
    assert {:ok, pid} = Server.start_link(new_id())
    assert 0 = Server.get_count(pid)
  end

  test "incrementing" do
    assert {:ok, pid} = Server.start_link(new_id())
    :ok = Server.increment(pid)
    assert 1 = Server.get_count(pid)
  end
end

OK, so all these tests are passing. Let's do one more tiny thing - we don't want to have to think about talking to the supervisor to start these. Let's make it so we can just say CountChocula.start_counter(some_id):

vim test/count_chocula_test.exs
defmodule CountChoculaTest do
  use ExUnit.Case
  import TestHelper

  test "starting a new counter" do
    assert {:ok, _pid} = CountChocula.start_counter(new_id())
  end
end

We can make this work by just delegating that function to the supervisor:

vim lib/count_chocula.ex
defmodule CountChocula do
  use Application

  def start(_type, _args) do
    import Supervisor.Spec, warn: false

    children = [
      supervisor(CountChocula.Supervisor, [])
    ]

    opts = [strategy: :one_for_one]
    Supervisor.start_link(children, opts)
  end

  defdelegate start_counter(id), to: CountChocula.Supervisor
end

With that, we can trivially start up counters and interact with them, and we have an umbrella application.

We're going to stop there for the moment. If you were building this with Ecto and Phoenix, you'd probably be done by now, right? Just add a row, use its ID, add CRUD operations to it with a generator, and update the row in the database. So what's the point?

Summary

I'm going to leave you with that question, until the next episode. Nothing we saw here was anything we haven't been doing for ages in this series, except perhaps the global registration bits, and I don't think I've covered umbrella apps before. So you might think this is a waste of your time. I hope you'll bear with me for one more episode though.

In the meantime, there are a lot of posts detailing the struggles that Rails developers ran into related to putting all of their application logic inside their Rails application directly. I even found out after starting this episode that Lance Halvorsen gave a talk at ElixirConfEU with exactly the same title I initially used, so after discussing potential confusion with him I tweaked it.. I've linked to quite a few of these posts in the resources section. I think it's vital that the Elixir Community avoids the pitfalls that plagued the Ruby community in this regard. We have better tools at our disposal - let's not hamstring ourselves! See you soon.

Resources