Category: Elixir

Elixir’s Getting Started guides go over the &Module.function/arity and &(...) anonymous function shorthand, but there are a couple neat tricks that are not immediately apparent about this shorthand.

For example, you can do something like &"...".

iex> hello_fun = &"Hello, #{&1}"
iex> hello_fun.("Keita")
"Hello, Keita"

Let’s have some more fun.

iex> fun = &~r/hello #{&1}/
iex> fun.("world")
~r/hello world/

iex> fun = &~w(hello #{&1})
iex> fun.("world")
["hello", "world"]
iex> fun.("world moon mars")
["hello", "world", "moon", "mars"]

iex> fun = &if(&1, do: "ok", else: "not ok")
iex> fun.(true)
"ok"
iex> fun.(false)
"not ok"

You can even use defmodule to create an anonymous function that defines a new module.

iex> fun = &defmodule(&1, do: def(hello, do: unquote(&2)))
iex> fun.(Hello, "hello there")
{:module, Hello, <<...>>, {:hello, 0}}
iex> Hello.hello
"hello there"

(Note that I don’t recommend overusing it like I did here! The only one that has been really useful to me was the first example, &"...")

In the beginning of 2015, I wrote a blog post about how my then-current programming language of choice (Ruby) was showing itself to not be as future-proof as I would have liked it to be.

A lot has changed since then, but a lot has remained the same.

First: I have started a few open-source Elixir projects:

• Exfile — a file upload handling, persistence, and processing library. Extracted from an image upload service I’m working on (also in Elixir).
• multistream-downloader — a quick tool to monitor and download HTTP-based live streams.
• runroller — a redirect “unroller” API, see the blog post about it.

The initial push to get me in to Elixir was indeed its performance, but that’s not what kept me. At the same time, I also tried learning Go and more recently, Rust has caught my attention.

In most cases, languages like Go and Rust can push more raw performance out of a simple “Hello World” benchmark test — exactly what I initially did to compare Ruby on Rails to Elixir / Phoenix. The more I used Elixir, the more I gained an appreciation for what I now regard critical language features (yes, most of these apply to Erlang and other functional languages too).

Immutability

This is a big one. Manipulating a mutable data structure is essentially changing the data in memory directly. This, however, is susceptible to the classic race condition bug programmers encounter when writing multithreaded programs:

• Thread 1 reads “1” from “a”
• Thread 2 reads “1” from “a”
• T1 increments “1” to “2” and writes it to “a”
• T2 increments “1” to “2” and writes it to “a”

In this case, the intended value is “3” because the programmer incremented “a” two times, but the actual value in memory is “2” due to the race condition.

In systems with immutable variables, this class of bug doesn’t exist. It forces the programmer to be explicit about shared state.

Lightweight Processes

A typical Erlang (Elixir) node can have millions of processes running on it. No, these are not your traditional OS processes — they are not threads, either. The Erlang VM uses its own scheduler and thread pool to execute code. The memory overhead of a single process is usually very light — on my machine, it’s 2.6kb (SMP and HiPE enabled).

Inter-Process Communication

Another big one.

Want to send a message to another process?

send(pid, :hello)

Want to handle a message from another process?

receive do
  :hello -> puts "I received :hello"
end

This works on processes that are running in the current node — but it also works across nodes. The syntax is exactly the same. The “pid” variable in the example is able to refer to a process anywhere in the cluster of nodes. The other node doesn’t even have to be Erlang, it just needs to be able to speak the same language, the distribution protocol and the external term format.

OTP

You can’t talk about Erlang or Elixir without bringing up OTP. OTP stands for “Open Telecom Platform” (Erlang was initially developed by Ericsson for use in telecom systems). OTP is a framework with many battle-tested tools to help you build your application — for example,

• gen_server – an abstraction of the client-server model
• gen_fsm – a finite state machine
• supervisor – a supervisor process that automates recovery from temporary failures

OTP is, for all intents and purposes, part of the Erlang standard library. Thus, it is automatically included in any Elixir application as well. The nuts and bolts of OTP are out of the scope of this blog post, but having such a rich toolbox is like a breath of fresh air coming from Ruby (I thought the same when switching full-time from PHP to Ruby).

Conclusion

I’ve learned a lot in this past year, and yet I feel like I’ve only scratched the surface. Thanks for reading!

I attended my first Elixir-related meetup yesterday, tokyo.ex #1.

(If the slides don’t work here, I have also uploaded them to YouTube.)

In my 5-minute lightning talk, I talked about the basics of using Exfile. Exfile is a file upload persistence and processing library for Elixir with integrations for Ecto and Phoenix.

In Ruby, there is a very handy class called StringIO. Basically, it allows you to treat a string like you would an IO object, such as an open file, etc. Very useful for in-memory “files” that you may not want to write to a temporary file.

In Elixir, there is a module called StringIO in the standard library. At first glance, these seem pretty similar:

Ruby:

Pseudo I/O on String object.

Elixir:

This module provides an IO device that wraps a string.

However, there are some subtle differences. For example, you can’t rewind the position of an Elixir StringIO:

iex(1)> {:ok, io} = StringIO.open("foo")
{:ok, #PID<0.59.0>}
iex(2)> IO.read(io, :all)
"foo"
iex(3)> :file.position(io, :bof)
===> hang!

(For more information about the :file.position/2 function, check the docs out. :bof stands for “beginning of file”)

Let’s see why this is happening. StringIO has a function to show its current buffers:

iex(1)> {:ok, io} = StringIO.open("foo")
{:ok, #PID<0.59.0>}
iex(2)> StringIO.contents(io)
{"foo", ""}
iex(3)> IO.read(io, :all)
"foo"
iex(4)> StringIO.contents(io)
{"", ""}

Basically, after you read data from an Elixir StringIO, it’s gone. So, we look to Erlang. Erlang’s file:open/2 function accepts a ram option that we might be interested in:

Returns an fd() which lets the file module operate on the data in-memory as if it is a file.

Let’s try it out.

iex(1)> {:ok, io} = :file.open("foo", [:ram, :binary])
{:ok, {:file_descriptor, :ram_file, #Port<0.1471>}}
iex(2)> IO.binread(io, :all)
"foo"
iex(3)> IO.binread(io, :all)
""
iex(4)> :file.position(io, :bof)
{:ok, 0}
iex(5)> IO.binread(io, :all)
"foo"

Rewinding works now.

Note that because of differences between Elixir’s IO / File modules and Erlang’s file module (probably related to how Elixir works with character encodings), you have to use the binary option to :file.open/2 and the bin-prefixed functions in Elixir land.

filter_zed_by = "1"
list = [
  %{"a" => "1", "b" => "2"},
  %{"a" => "1", "b" => "5"},
  %{"a" => "2", "b" => "5"},
  %{"z" => "1", "x" => "2"}
]
Enum.filter list, fn
  %{"z" => ^filter_zed_by} -> true
  _ -> false
end

# => [%{"z" => "1", "x" => "2"}]

• case in a fun is usually redundant
• if is even worse
• keep it simple

Edit 2016/4/29 I have written a follow-up piece to this blog post.

As many of you probably know, I am a professional programmer. I started my professional career with WordPress and PHP development, and now I find myself doing a lot of Ruby work. I am still in the very early stages of my professional career — I have only been doing this for about 5 years. There are people who are much more experienced than I am, and there is a whole world of things that I have yet to learn and experience.

Until recently, I’ve been relatively reluctant to change. I just wanted to get something done, using the tools I’m most familiar with. In the first few years of my career, that was PHP. After learning and getting used to Ruby and Rails, that’s been my go-to language and framework.

However, I’ve decided to throw caution in the wind, and choose a new language that I’m going to be using going forward: Elixir.

Every programming language has something that irks somebody. The thing that irks me about Ruby is its lack of a really robust concurrency model. There have been experiments in EventMachine — evented I/O like Node, Rubinius and JRuby — real threads, Unicorn — forking (!). But none of these solutions are particularly elegant. Relying on forking for concurrency, even with copy-on-write, is not efficient at all. (Read more about concurrency and Ruby: Matz is not a threading guy)

Enter Elixir. Elixir is a language that runs on top of the Erlang VM. The Erlang environment is something special, I think. It’s built from the ground up with concurrency and reliability in mind (it was developed for use in telecom systems). Originally a proprietary language at Ericsson for a little over 10 years, it was open-sourced in 1998. Elixir builds on this very mature ecosystem, and makes it accessible enough for “mere mortals” to build high-performance applications.

When I first tried Elixir out to build a web app, like you would with Rails, I was pretty surprised. If you’re used to Rails, you’ve seen this log message:

This is the Elixir / Phoenix framework equivalent:

170ms down to 5ms. And that’s in development mode. Switch it to production mode, and this is what you see:

5ms down to 300 microseconds.

Granted, this is a very simple page. It’s basically a “Hello World” benchmark, but it certainly piqued my interest. This amount of performance, coupled with the ability to maintain it even under immense load, is one of the factors going in to my choice of Elixir.

The other factor is, what I believe, the direction of the Internet. We’re moving away from servers rendering static HTML and sending that over the wire, and to more dynamic content powered by client-side JavaScript or native code on mobile devices. Elixir and Erlang handle this beautifully — dutifully keeping thousands and thousands of open WebSocket sessions alive without breaking a sweat. You should take a look at how Whatsapp uses one machine running their chat backend on Erlang to service more than 2 million simultaneous connections.

By investing my time now in Elixir and Erlang, I believe that I’m essentially getting ready for the next 20 years of the Internet. And I’m excited.