Elixir's StringIO may not be what you think it is

January 23, 2016

In Ruby, there is a very handy class called StringIO. Basically, it allows you to treat a string like you would an IO object, such as an open file, etc. Very useful for in-memory “files” that you may not want to write to a temporary file.

In Elixir, there is a module called StringIO in the standard library. At first glance, these seem pretty similar:

Ruby:

Pseudo I/O on String object.

Elixir:

This module provides an IO device that wraps a string.

However, there are some subtle differences. For example, you can’t rewind the position of an Elixir StringIO:

iex(1)> {:ok, io} = StringIO.open("foo")
{:ok, #PID<0.59.0>}
iex(2)> IO.read(io, :all)
"foo"
iex(3)> :file.position(io, :bof)
===> hang!

(For more information about the :file.position/2 function, check the docs out. :bof stands for “beginning of file”)

Let’s see why this is happening. StringIO has a function to show its current buffers:

iex(1)> {:ok, io} = StringIO.open("foo")
{:ok, #PID<0.59.0>}
iex(2)> StringIO.contents(io)
{"foo", ""}
iex(3)> IO.read(io, :all)
"foo"
iex(4)> StringIO.contents(io)
{"", ""}

Basically, after you read data from an Elixir StringIO, it’s gone. So, we look to Erlang. Erlang’s file:open/2 function accepts a ram option that we might be interested in:

Returns an fd() which lets the file module operate on the data in-memory as if it is a file.

Let’s try it out.

iex(1)> {:ok, io} = :file.open("foo", [:ram, :binary])
{:ok, {:file_descriptor, :ram_file, #Port<0.1471>}}
iex(2)> IO.binread(io, :all)
"foo"
iex(3)> IO.binread(io, :all)
""
iex(4)> :file.position(io, :bof)
{:ok, 0}
iex(5)> IO.binread(io, :all)
"foo"

Rewinding works now.

Note that because of differences between Elixir’s IO / File modules and Erlang’s file module (probably related to how Elixir works with character encodings), you have to use the binary option to :file.open/2 and the bin-prefixed functions in Elixir land.