Subscribe now

Preparing to Learn About Phoenix [04.25.2017]

Last week we learned a lot about Ecto, the most common way to interact with your database in Elixir. This week we'll learn about Phoenix, the most popular web framework. Phoenix offers great conventions for building web applications that handle ridiculously high amounts of traffic without breaking a sweat, thanks to the underlying Erlang Virtual Machine. Before we dig in, let's look at a solution to last week's exercise.

Exercise Solution

I've started with the dailydrip/firestorm repo tagged before_episode_005.1.

We had a few basic goals:

  • Given a category, find the three threads with the most recent posts.
  • Given a user, find all of their posts.
  • Given a user, find all of the threads in which they have posted.
  • Given a thread, return the number of posts.
  • Find all posts whose body contains a particular string.

We'll build these out with tests. Let's start at the beginning. We know we want to generate a few threads and posts, and we don't much care about the content. We can write our own functions to do this for us, but I tend to prefer some kind of factory for this, and the fine folks at Thoughtbot have created ex_machina for this use case, so let's use it. We'll also bring in Faker to create easy fake data:

cd apps/firestorm_data
vim mix.exs
defmodule FirestormData.Mixfile do
  # ...
  defp deps do
    [
      # ...
      {:ex_machina, "~> 2.0", only: :test},
      {:faker, "~> 0.7", only: :test},
    ]
  end
  # ...
end
mix deps.get

Now we need to make sure that this gets started with our tests, so we'll update test/test_helper.exs:

ExUnit.start()
Ecto.Adapters.SQL.Sandbox.mode(FirestormData.Repo, :manual)
# Add this
{:ok, _} = Application.ensure_all_started(:ex_machina)

Also, we need to make sure that our factories are loaded when we run our tests. We'll put them under test/support. In a Phoenix application this is already in our compilation path, but since this is just a normal Elixir application so far we need to add it. This also goes in mix.exs:

defmodule FirestormData.Mixfile do
  def project do
    [
      # ...
      elixirc_paths: elixirc_paths(Mix.env),
    ]
  end

  # This makes sure your factory and any other modules in test/support are compiled
  # when in the test environment.
  defp elixirc_paths(:test), do: ["lib", "web", "test/support"]
  defp elixirc_paths(_), do: ["lib", "web"]
  # ...
end

We'll add a factory to produce data for all of our schemas, using Faker to make some decent approximation of the right shape of data.

mkdir test/support
vim test/support/factory.ex
defmodule FirestormData.Factory do
  use ExMachina.Ecto, repo: FirestormData.Repo
  alias FirestormData.{Category, Thread, Post, User}

  def category_factory() do
    %Category{
      title: Faker.Lorem.sentence(%Range{first: 1, last: 10}),
    }
  end

  def thread_factory() do
    %Thread{
      title: Faker.Lorem.sentence(%Range{first: 1, last: 10}),
      category: build(:category),
    }
  end

  def user_factory() do
    %User{
      name: Faker.Name.name,
      username: Faker.Internet.user_name,
      email: Faker.Internet.email,
    }
  end

  def post_factory() do
    %Post{
      body: Faker.Lorem.paragraph,
      user: build(:user),
      thread: build(:thread),
    }
  end
end

Finding the most recently active threads in a category

Now let's make a category withs some threads and posts and ensure that the most recently posted threads show up at the top of a new query. First, we'll create the data in our test:

defmodule FirestormData.ThreadTest do
  alias FirestormData.{Category, Thread, Repo}
  import FirestormData.Factory
  use ExUnit.Case
  # ...
  # Lots of sample data insertion happening here
  test "finding the three threads with the most recent posts in a given category" do
    category = insert(:category)
    threads = insert_list(5, :thread, %{category: category})
    for thread <- threads do
      insert_list(3, :post, %{thread: thread})
    end
    [thread1, thread2, thread3, thread4, thread5] = threads
    :timer.sleep 1
    insert(:post, %{thread: thread1})
    insert(:post, %{thread: thread3})
    insert(:post, %{thread: thread5})
    other_thread_post = insert(:post)

    recent_threads =
      category
      |> Thread.get_recent_threads
      |> Repo.all

    thread_ids =
      recent_threads
      |> Enum.map(&(&1.id))

    assert thread1.id in thread_ids
    assert thread3.id in thread_ids
    assert thread5.id in thread_ids
    refute thread2.id in thread_ids
    refute thread4.id in thread_ids
    refute other_thread_post.thread_id in thread_ids
  end
end

This explains what we want. Our first attempt at this query is as follows:

defmodule FirestormData.Thread do
  use Ecto.Schema
  import Ecto.Changeset
  # ...
  # Queries - we should move these out of the schema, but we'll put them here
  # for now for illustrative purposes.
  def get_recent_threads(category) do
    alias FirestormData.{Post, Repo}
    import Ecto.Query
    from p in Post,
      order_by: [desc: p.inserted_at],
      left_join: t in __MODULE__, on: [id: p.thread_id],
      where: t.category_id == ^category.id,
      limit: 3,
      distinct: t.id,
      select: t
  end
end

This won't return what we want because our distinct: t.id is actually forcing SQL to order by the thread id as well, initially...this is the wrong way to write this query. But it shows lots of ecto stuff we haven't covered previously!

It turns out this query is super hard. Oops! I should have thought about that. Anyway, here's a LEFT LATERAL JOIN version that will get us what we want - I actually had to dig around a bit for some SQL and ended up on this ElixirForum post without using Elixir in the search term :)

  # Queries - we should move these out of the schema, but we'll put them here
  # for now for illustrative purposes.
  def get_recent_threads(category) do
    import Ecto.Query

    __MODULE__
      |> join(:left_lateral, [t], p in fragment("SELECT thread_id, inserted_at FROM posts WHERE posts.thread_id = ? ORDER BY posts.inserted_at DESC LIMIT 1", t.id))
      |> order_by([t, p], [desc: p.inserted_at])
      |> where(category_id: ^category.id)
      |> select([t], t)
      |> limit(3)
  end

This should make the test pass. Sorry, I didn't mean for it to be a SQL exercise but that's how it ended up!

Also, I used the pipeline version of Ecto queries here. I'm still trying to use these more frequently, so I might shift between the from form and the pipelines from time to time. Think of it as a learning exercise rather than a wholly inconsistent host! Also - if you find a nicer way to do this I'm all ears. Hit up the comments.

Find a user's posts

Next, we'd like to find a User's posts. This is easy. First, we'll write the test:

defmodule FirestormData.PostTest do
  alias FirestormData.{Category, User, Thread, Post, Repo}
  use ExUnit.Case
  import Ecto.Query
  import FirestormData.Factory
  # ...
  test "finding a user's Posts", %{josh: josh} do
    [post1, post2, post3] = insert_list(3, :post, %{user: josh})
    [post4, post5, post6] = insert_list(3, :post)

    post_ids =
      josh
      |> Post.for_user
      |> Repo.all
      |> Enum.map(&(&1.id))

    assert post1.id in post_ids
    assert post2.id in post_ids
    assert post3.id in post_ids
    refute post4.id in post_ids
    refute post5.id in post_ids
    refute post6.id in post_ids
  end
  # ...
end

This won't work yet because there is no Post.for_user/1 function. Let's add it:

defmodule FirestormData.Post do
  # ...
  def for_user(user) do
    import Ecto.Query

    __MODULE__
    |> where([p], p.user_id == ^user.id)
  end
end

Now we can find a user's posts easily. Of course, we could have done this with the reverse association as well, but this is just a basic query and it makes me happy.

Find threads a user has posted in

Next we'll find threads a user has posted in. Again, fairly easy. First, the test:

defmodule FirestormData.ThreadTest do
  # ...
  test "find threads a user has posted in" do
    user = insert(:user)
    [thread1, thread2, thread3] = insert_list(3, :thread)
    insert(:post, %{thread: thread1, user: user})
    insert(:post, %{thread: thread3, user: user})
    insert(:post, %{thread: thread2})

    user_thread_ids =
      user
      |> Thread.posted_in_by_user
      |> Repo.all
      |> Enum.map(&(&1.id))

    assert thread1.id in user_thread_ids
    assert thread3.id in user_thread_ids
    refute thread2.id in user_thread_ids
  end
end

Next, we'll create the function:

defmodule FirestormData.Thread do
  # ...
  def posted_in_by_user(user) do
    import Ecto.Query
    alias FirestormData.{Post, Thread}

    Post
      |> where([p], p.user_id == ^user.id)
      |> join(:inner, [p], t in Thread, p.thread_id == t.id)
      |> select([p, t], t)
  end
end

The only quirky thing here is how we aliased the module we were in - I couldn't use __MODULE__ inside the Ecto macro, so I had to resort to that trick. Anyway, this is again just some basic SQL done Ecto-style. I know we haven't gone over it all in detail, but I'm hoping that seeing some examples serves to explain it a bit better than long winded text explanations will.

Given a thread, return the number of posts.

Now we'd like to see the number of posts in a thread. That's easy:

  test "find out how many posts there are in a thread" do
    thread = insert(:thread)
    insert_list(5, :post, %{thread: thread})

    assert 5 = thread |> Thread.post_count |> Repo.one
  end
  def post_count(thread) do
    import Ecto.Query
    alias FirestormData.Post

    Post
      |> where([p], p.thread_id == ^thread.id)
      |> group_by([p], p.thread_id)
      |> select([p], count(p.id))
  end

Find all posts whose body contains a particular string

There are a lot of ways to find things containing some text. The way we'll do it right now is not remotely as efficient as you can get. We'll eventually use PostgreSQL's TSVectors for Firestorm search, but for now we'll just use ILIKE:

defmodule FirestormData.PostTest do
  # ...
  test "finding posts with a particular string in them" do
    post1 = insert(:post, %{body: "bar banana foo"})
    post2 = insert(:post, %{body: "bar potato foo"})
    post3 = insert(:post, %{body: "bar zanzibar foo"})

    post_ids =
      "banana"
      |> Post.containing_body
      |> Repo.all
      |> Enum.map(&(&1.id))

    assert [post1.id] == post_ids
  end
  # ...
end
defmodule FirestormData.Post do
  # ...
  def containing_body(body) do
    import Ecto.Query

    __MODULE__
    |> where([p], ilike(p.body, ^"%#{body}%"))
  end
end

Here we see a particularly weird looking query - we have to actually pin the string rather than pinning inside the interpolation. I'm not sure exactly why this is, but Ecto's nice enough to tell you if you do it wrong.

Exercise Summary

In today's episode we saw how to create a host of different Ecto queries to find out more interesting data for Firestorm. It's a bit dense, but I wanted to cover a lot of ground that I couldn't cover last week in detail.

Preparatory Readings

Now we can prepare for a week on Phoenix. We'll start a new Phoenix app tomorrow, but that means you should go ahead and install its mix task today. You can do this with:

# Since Phoenix 1.3 hasn't been released yet, we'll install its generator
# directly
mix archive.install https://github.com/phoenixframework/archives/raw/master/1.3-rc/phx_new-1.3.0-rc.1.ez
# Once Phoenix 1.3 gets released, you can use this instead
# mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez

This installs the master installer. If you want a particular release, you can tweak the path you give to mix archive.install. This should work fine for now. For reference, as of this writing the second release candidate has been released for Phoenix 1.3.

The Phoenix website has some fantastic docs and guides. I recommend checking those out if you want to get a head start. You'll also want to make sure you have node installed for the frontend assets. See you soon!

Resources