Elixir’s StringIO may not be what you think it is

In Ruby, there is a very handy class called StringIO. Basically, it allows you to treat a string like you would an IO object, such as an open file, etc. Very useful for in-memory “files” that you may not want to write to a temporary file.

In Elixir, there is a module called StringIO in the standard library. At first glance, these seem pretty similar:

Ruby:

Pseudo I/O on String object.

Elixir:

This module provides an IO device that wraps a string.

However, there are some subtle differences. For example, you can’t rewind the position of an Elixir StringIO:

iex(1)> {:ok, io} = StringIO.open("foo")
{:ok, #PID<0.59.0>}
iex(2)> IO.read(io, :all)
"foo"
iex(3)> :file.position(io, :bof)
===> hang!

(For more information about the :file.position/2 function, check the docs out. :bof stands for “beginning of file”)

Let’s see why this is happening. StringIO has a function to show its current buffers:

iex(1)> {:ok, io} = StringIO.open("foo")
{:ok, #PID<0.59.0>}
iex(2)> StringIO.contents(io)
{"foo", ""}
iex(3)> IO.read(io, :all)
"foo"
iex(4)> StringIO.contents(io)
{"", ""}

Basically, after you read data from an Elixir StringIO, it’s gone. So, we look to Erlang. Erlang’s file:open/2 function accepts a ram option that we might be interested in:

Returns an fd() which lets the file module operate on the data in-memory as if it is a file.

Let’s try it out.

iex(1)> {:ok, io} = :file.open("foo", [:ram, :binary])
{:ok, {:file_descriptor, :ram_file, #Port<0.1471>}}
iex(2)> IO.binread(io, :all)
"foo"
iex(3)> IO.binread(io, :all)
""
iex(4)> :file.position(io, :bof)
{:ok, 0}
iex(5)> IO.binread(io, :all)
"foo"

Rewinding works now.

Note that because of differences between Elixir’s IO / File modules and Erlang’s file module (probably related to how Elixir works with character encodings), you have to use the binary option to :file.open/2 and the bin-prefixed functions in Elixir land.

One thought on “Elixir’s StringIO may not be what you think it is”

  1. Thank you for your post.

    Also, there’s one more thing that the RAM file is superior than StringIO is currently StringIO doesn’t support binary mode, i.e. for writing/reading binary.

Leave a Reply

Your email address will not be published. Required fields are marked *