[252] Elixir 1.3 Mix Tasks

Checking out the new mix tasks provided in Elixir 1.3: `xref`, `app.tree`, `deps.tree`, `test --stale`, and `escript.install`

Subscribe now

Elixir 1.3 Mix Tasks [06.27.2016]

Elixir 1.3 came out this week, and it has a host of great new features. I wanted to call out some of the new mix tasks, because they're kind of awesome. Let's have a look.

Project

We'll use the elixir codebase itself for all of this. Let's get straight into it!

xref

There's a new xref task that performs cross-reference checks between modules. Let's look at its help:

mix help xref
Performs cross reference checks between modules.

Xref modes

The xref task expects a mode as first argument:

┃ mix xref MODE

All available modes are discussed below.

warnings

Prints warnings for violated cross reference checks:

┃ mix xref warnings

This is the mode used by Mix during compilation.

unreachable

Prints all unreachable "file:line: module.function/arity" entries:

┃ mix xref unreachable

The "file:line" represents the file and line a call to an unknown
"module.function/arity" is made.

callers CALLEE

Prints all callers of the given CALLEE, which can be one of: Module,
Module.function, or Module.function/arity. Examples:

┃ mix xref callers MyMod
┃ mix xref callers MyMod.fun
┃ mix xref callers MyMod.fun/3

graph

Prints a file dependency graph where an edge from A to B indicates that A
depends on B.

┃ mix xref graph --format dot

The following options are accepted:

  • --exclude - paths to exclude
  • --source - display all files that the given source file references
    (directly or indirectly)
  • --sink - display all files that reference the given file (directly or
    indirectly)
  • --format - can be set to one of:
    • pretty - use Unicode codepoints for formatting the graph. This is
      the default except on Windows

    • plain - do not use Unicode codepoints for formatting the graph.
      This is the default on Windows

    • dot - produces a DOT graph description in xref_graph.dot in the
      current directory. Warning: this will override any previously generated
      file


The --source and --sink options are particularly useful when trying to
understand how the modules in a particular file interact with the whole system.

Shared options

Those options are shared across all modes:

  • --no-compile - do not compile even if files require compilation
  • --no-deps-check - do not check dependencies
  • --no-archives-check - do not check archives
  • --no-elixir-version-check - do not check the Elixir version from
    mix.exs

Configuration

All configuration for Xref should be placed under the key :xref.

  • :exclude - a list of modules and {module, function, arity} tuples to
    ignore when checking cross references. For example: [MissingModule,
    {MissingModule2, :missing_func, 2}]

So we have a few modes it supports, and we'll check them all out:

  • warnings
  • unreachable
  • callers
  • graph

warnings

This will print out warnings regarding cross-reference checks in a given codebase.

mix xref warnings
warning: function :gen.debug_options/2 is undefined or private
  lib/gen_event.ex:653

Let's use the next mode to help us understand this.

unreachable

xref's unreachable is supposed to tell you which bits of code in a project aren't reachable.

cd ~/elixir/elixir/lib/elixir
mix xref unreachable
lib/gen_event.ex:653: :gen.debug_options/2

So apparently that line isn't reachable. Guess it's not a big deal then!

callers

The callers mode will tell you who calls a given module or function. Let's see what other modules call into the String module:

mix xref callers String
lib/module.ex:1016: String.split/2
lib/module.ex:649: String.to_atom/1
lib/module.ex:655: String.to_atom/1
lib/io/ansi/docs.ex:157: String.duplicate/2
lib/io/ansi/docs.ex:335: String.duplicate/2
lib/io/ansi/docs.ex:45: String.length/1
lib/io/ansi/docs.ex:157: String.length/1
lib/io/ansi/docs.ex:413: String.next_grapheme/1
lib/io/ansi/docs.ex:46: String.pad_leading/2
lib/io/ansi/docs.ex:46: String.pad_trailing/2
lib/io/ansi/docs.ex:296: String.replace/3
lib/io/ansi/docs.ex:426: String.replace/3
lib/io/ansi/docs.ex:214: String.split/2
lib/io/ansi/docs.ex:290: String.split/2
lib/io/ansi/docs.ex:60: String.split/3
lib/io/ansi/docs.ex:71: String.trim/1
lib/io/ansi/docs.ex:77: String.trim/1
lib/io/ansi/docs.ex:83: String.trim/1
lib/io/ansi/docs.ex:289: String.trim/1
lib/io/ansi/docs.ex:288: String.trim/2
lib/io/ansi/docs.ex:61: String.trim_trailing/1
lib/io/ansi/docs.ex:122: String.upcase/1
lib/kernel/typespec.ex:699: String.downcase/1
lib/kernel/typespec.ex:701: String.downcase/1
lib/kernel/typespec.ex:699: String.to_atom/1
lib/kernel/typespec.ex:701: String.to_atom/1
lib/option_parser.ex:361: String.replace/3
lib/option_parser.ex:588: String.to_atom/1
lib/option_parser.ex:377: String.trim_leading/2
lib/option_parser.ex:398: String.trim_leading/2
lib/option_parser.ex:618: String.trim_leading/2
lib/kernel/cli.ex:327: String.to_atom/1
lib/kernel/cli.ex:338: String.to_atom/1
lib/base.ex:146: String.duplicate/2
lib/string.ex:2058: String.to_charlist/1
lib/record/extractor.ex:30: String.to_charlist/1
lib/record/extractor.ex:39: String.to_charlist/1
lib/version.ex:429: String.split/2
lib/kernel.ex:4257: String.split/1
lib/kernel.ex:3131: String.to_atom/1
lib/kernel.ex:3168: String.to_atom/1
lib/kernel.ex:4260: String.to_atom/1
lib/kernel.ex:4052: String.to_charlist/1
lib/kernel.ex:4078: String.to_charlist/1
lib/kernel.ex:4261: String.to_charlist/1
lib/calendar/iso.ex:152: String.duplicate/2
lib/system.ex:345: String.split/3
lib/system.ex:329: String.to_charlist/1
lib/system.ex:359: String.to_charlist/1
lib/system.ex:383: String.to_charlist/1
lib/system.ex:405: String.to_charlist/1
lib/system.ex:458: String.to_charlist/1
lib/system.ex:538: String.to_charlist/1
lib/system.ex:600: String.to_charlist/1
lib/system.ex:602: String.to_charlist/1
lib/kernel/parallel_compiler.ex:271: String.length/1
lib/kernel/parallel_compiler.ex:275: String.pad_leading/2
lib/protocol.ex:26: String.to_atom/1
lib/inspect.ex:145: String.printable?/1
lib/inspect.ex:201: String.printable?/1
lib/inspect.ex:454: String.printable?/1
lib/uri.ex:555: String.contains?/2
lib/uri.ex:410: String.downcase/1
lib/uri.ex:513: String.split/2
lib/uri.ex:425: String.trim_leading/2
lib/uri.ex:425: String.trim_trailing/2
lib/exception.ex:661: String.jaro_distance/2
lib/exception.ex:434: String.split/2
lib/list/chars.ex:30: String.to_charlist/1
lib/code.ex:622: String.to_charlist/1

You can also ask for a particular function. Let's look for calls to to_atom/1:

mix xref callers String.to_atom/1
lib/module.ex:649: String.to_atom/1
lib/module.ex:655: String.to_atom/1
lib/kernel/typespec.ex:699: String.to_atom/1
lib/kernel/typespec.ex:701: String.to_atom/1
lib/option_parser.ex:588: String.to_atom/1
lib/kernel/cli.ex:327: String.to_atom/1
lib/kernel/cli.ex:338: String.to_atom/1
lib/kernel.ex:3131: String.to_atom/1
lib/kernel.ex:3168: String.to_atom/1
lib/kernel.ex:4260: String.to_atom/1
lib/protocol.ex:26: String.to_atom/1

You can imagine how useful getting at this sort of data is, for building tools especially. I expect most IDEs to start to make use of this data now.

It's also possible to ask without specifying the arity.

graph

You can generate graphs of cross-references between modules. To see the graph in text, you can just type mix xref graph.

# omitting part of this because it's huge
lib/string/chars.ex
├── lib/atom.ex (compile)
│   ├── lib/kernel.ex (compile)
│   │   ├── lib/access.ex (compile)
│   │   │   ├── lib/kernel.ex (compile)
│   │   │   ├── lib/kernel/typespec.ex (compile)
│   │   │   │   ├── lib/kernel.ex (compile)
│   │   │   │   ├── lib/module.ex (compile)
│   │   │   │   │   ├── lib/kernel.ex (compile)
│   │   │   │   │   ├── lib/kernel/typespec.ex (compile)
│   │   │   │   │   ├── lib/macro/env.ex (compile)
│   │   │   │   │   │   ├── lib/kernel.ex (compile)
│   │   │   │   │   │   ├── lib/kernel/typespec.ex (compile)
│   │   │   │   │   │   ├── lib/keyword.ex (compile)
│   │   │   │   │   │   │   ├── lib/enum.ex (compile)
│   │   │   │   │   │   │   │   ├── lib/collectable.ex (compile)
│   │   │   │   │   │   │   │   │   ├── lib/kernel.ex (compile)
│   │   │   │   │   │   │   │   │   ├── lib/kernel/typespec.ex (compile)
│   │   │   │   │   │   │   │   │   ├── lib/list.ex (compile)
│   │   │   │   │   │   │   │   │   │   ├── lib/kernel.ex (compile)
│   │   │   │   │   │   │   │   │   │   ├── lib/kernel/typespec.ex (compile)
│   │   │   │   │   │   │   │   │   │   ├── lib/module.ex (compile)
│   │   │   │   │   │   │   │   │   │   ├── lib/string.ex (compile)
# ...

Of course that might not be so easy to follow. To see just those pieces that reference a given file, you can use the sink option:

mix xref graph --sink lib/string.ex
lib/string/chars.ex
├── lib/code.ex
│   ├── lib/path.ex
│   │   ├── lib/exception.ex
│   │   │   ├── lib/path.ex
│   │   │   ├── lib/process.ex
│   │   │   │   ├── lib/kernel.ex (compile)
│   │   │   │   │   ├── lib/macro.ex
│   │   │   │   │   │   ├── lib/list.ex
│   │   │   │   │   │   │   ├── lib/exception.ex
│   │   │   │   │   │   ├── lib/exception.ex
│   │   │   │   │   │   ├── lib/inspect.ex
│   │   │   │   │   │   │   ├── lib/code.ex
│   │   │   │   │   │   │   ├── lib/macro.ex (compile)
│   │   │   │   │   │   │   ├── lib/list.ex (compile)
│   │   │   │   │   │   │   ├── lib/exception.ex
│   │   │   │   │   │   │   ├── lib/protocol.ex (compile)
│   │   │   │   │   │   │   │   ├── lib/code.ex
│   │   │   │   │   │   │   │   ├── lib/list.ex
│   │   │   │   │   │   │   │   ├── lib/list/chars.ex
│   │   │   │   │   │   │   │   │   ├── lib/code.ex
# ...

You can do that in the reverse as well, to just see what files a given file references, with source:

mix xref graph --source lib/string.ex

Of course, no one really wants to read this stuff in a text format probably, so you can generate graphviz data for the dependencies with the dot formatter:

mix xref graph --source lib/string.ex --format dot
dot -Tpng xref_graph.dot -o xref_graph.png
gnome-open xref_graph.png

And we can think of this as performance art perhaps in this case...I'm sure there are better examples of this being useful, but for now it's just really cool. I used to do this with my ruby projects to help me figure out dumb dependency chains, so I have first-hand experience of how it can help you make sense of your designs.

app.tree and deps.tree

The app.tree and deps.tree mix tasks are another way to visualize dependencies within your application.

For this piece, we'll use the phoenix source code.

cd ~/elixir/phoenix

app.tree

Let's look at the options for the app.tree task:

mix help app.tree
                                  mix app.tree

Prints the application tree.

┃ mix app.tree --exclude logger --exclude elixir

If no application is given, it uses the current application defined in the
mix.exs file.

Command line options

  • --exclude - exclude applications which you do not want to see printed.
    kernel, stdlib and compiler are always excluded from the tree.
  • --format - Can be set to one of either:
    • pretty - use Unicode codepoints for formatting the tree. This is
      the default except on Windows.

    • plain - do not use Unicode codepoints for formatting the tree. This
      is the default on Windows.

    • dot - produces a DOT graph description of the application tree in
      app_tree.dot in the current directory. Warning: this will override any
      previously generated file.

To see the app tree for a given application, just run mix app.tree. This will show the application dependencies:

mix app.tree

Here, too, you can generate a graphviz file:

mix app.tree --format dot
dot -Tpng app_tree.dot -o app_tree.png
gnome-open app_tree.png

This is a bit easier to follow, right?

deps.tree

You can also see the dependencies tree. We'll start with the help:

mix help deps.tree

Let's look at phoenix's dependencies:

mix deps.tree
phoenix
├── websocket_client (https://github.com/jeremyong/websocket_client.git)
├── gettext ~> 0.8 (Hex package)
├── poison ~> 1.5 or ~> 2.0 (Hex package)
├── ex_doc ~> 0.12 (Hex package)
│   └── earmark ~> 0.2 (Hex package)
├── phoenix_pubsub ~> 1.0 (Hex package)
├── cowboy ~> 1.0 (Hex package)
│   ├── cowlib ~> 1.0.0 (Hex package)
│   └── ranch ~> 1.0 (Hex package)
├── plug ~> 1.1 (Hex package)
│   └── cowboy ~> 1.0 (Hex package)
├── phoenix_html ~> 2.6 (Hex package)
│   └── plug ~> 0.13 or ~> 1.0 (Hex package)
└── inch_ex ~> 0.2 (Hex package)
    └── poison ~> 1.2 (Hex package)

And as graphviz:

mix deps.tree --format dot
dot -Tpng deps_tree.dot -o deps_tree.png
gnome-open deps_tree.png

Again, this is extremely useful.

test --stale

The test task had a stale option added that only runs tests that should have been affected by recent file changes. It can use the information from the xref bits to figure this out.

We'll run the phoenix tests once to start:

mix test

Now let's change something that isn't depended on by a lot of other files:

vim lib/mix/tasks/phoenix.gen.channel.ex
# add an IO.puts to a function
mix test --stale

See how it knew just to run that one set of tests? This is pure gold, and I expect should make TDD tons faster.

escript.install

There's also a new mix task for installing escripts. You can install them from local files or from repos, or a variety of other options:

mix help escript.install

Let's install one:

mix escript.install github dazuma/erl2ex

Now you can run it. If you added .mix/escripts to your PATH, you could invoke them by name. For now, I'll just run it with the path:

~/.mix/escripts/erl2ex --help

Cool! We got executables from an escript repo with one line. This is a much nicer experience than we've had before.

Summary

Today we just looked at the new mix tasks that were added in Elixir 1.3. The biggest part of this was the mechanisms introduced for identifying cross-references. This allowed a lot of really cool stuff, and should help make elixir's tooling even better over time. I hope you enjoyed it. See you soon!

Resources