Ruby, OpenGL, and Memory

Memory is a tricky topic in Ruby -- it's not much fun, but it's doable. Working with memory directly is often the opposite of what you want when you code in Ruby, but using OpenGL and other FFI-based libraries sometimes necessitates this without getting into complicated wrapper libraries. At least in opengl-core, you're going to end up doing some amount of memory management.

With that in mind, this document tries to cover a few ways to work with memory in Ruby. In particular, I'm going to try to cover two options for memory management when using opengl-core: Fiddle and snow-data.

If you use opengl-core, it makes two important assumptions:

1. You already know can learn the OpenGL API.

   This is doable, but assumes a bit of C knowledge. It's hard to get around with OpenGL without wrapper libraries, and opengl-core doesn't try to address this.

2. You know how to work with memory, either via Fiddle or some other gem that works with Fiddle.

   This assumption tends to be incorrect, because most people don't have to and probably don't want to think about memory in Ruby. This isn't good or bad, Ruby just tends to not have much focus on lower-level programming.

The first point is a given -- if you're a programmer, you're equipped to learn something. But the second point isn't quite as simple, because there's not a lot out there that really tries to show you how to work with memory in Ruby. Plus, most people aren't familiar with Ruby FFIs, including Fiddle, and it's not likely you'll just stumble across it when you need to work with memory.

So, this document is intended for people who may need a short introduction to working with memory in Ruby. It covers using both Fiddle, part of Ruby's stdlib, and snow-data, a library I wrote specifically for working with memory in OpenGL and Ruby.

All sizes and memory offsets in this document are measured in bytes. If you need a simple GL script to test things out with, try this gltest.rb gist. If the gist stops working, let me know and I'll do what I can to fix it.

Fiddle

Fiddle's a fairly simple libffi wrapper library. This means it means it deals in more than just memory, but we're just going to look at memory right now.

Fiddle provides a few tools for working with pointers, because a lot of C APIs will expect you to be able to use pointers. OpenGL in particular just about requires you to think about things using C's memory model.

First off, if you want to allocate a block of memory, use the malloc method:

require 'fiddle' # This line will be omitted in later examples

# Allocate a 1024-byte block, get a pointer in return.
ptr = Fiddle.malloc(1024)               # => 140660413834240

The pointer returned is a number -- just the address of the memory. Ruby's garbage collector isn't managing this memory, it's there until you explicitly free it. In other words, it's not safe, and you can't really do anything with just the address anyway. Still, it's important to free it:

Fiddle.free(ptr)                        # => nil

This isn't really the most convenient way to work with memory in Fiddle. So, instead, you can also use Fiddle::Pointer, which is just a simple wrapper around pointers. For example:

ptr = Fiddle::Pointer.malloc(1024)      # => #<Fiddle::Pointer:...>

This gives you a Pointer object to work with, which is a bit more convenient. That said, by default, Fiddle pointers have no free function attached. You can assign one, but it wants a Fiddle::Function for this. That's a bit beyond the scope of this document, though, so you'll want to refer to Fiddle's own documentation for more detail. So, by default, this memory is not actually freed when the GC collects the Fiddle::Pointer.

You can still just call Fiddle.free on the Pointer's address (obtained with to_i) and discard it, as shown below. Keep in mind, however, that this doesn't take care of copies of the pointer with the same address -- dangling pointers are as much an issue in Ruby as they are in C with this, so you need to exercise caution.

Fiddle.free(ptr.to_i)                   # => nil

The Pointer object gives you two ways to work with the memory you've allocated. First, you can pull an arbitrary unsigned 8-bit integer at an offset, or you can pull a sequence of integers as a string (the integers making up the characters of the string).

For example, given a pointer ptr, you can do things like the following:

ptr = Fiddle::Pointer.malloc(8)

# Assign some byte values
ptr[0] = 77                             # => 98
ptr[1] = 101                            # => 97
ptr[2] = 97                             # => 122

# Pull just three bytes out as a string
ptr[0, 3]                               # => "baz"

# Note: ranges do not work for extracting a string.

You could also use Array.pack and String.unpack to compact data into a block of memory, but this can be fairly expensive to perform (due to cost in time, at least at the time of this writing).

Instead, let's talk about using snow-data as an alternative.

snow-data

snow-data is a simple Ruby C extension that provides tools for working with memory, including defining C-like data structures, and reading and writing to memory.

First off, allocating memory with snow-data is functionally similar to using Fiddle:

require 'snow-data'

ptr = Snow::Memory.malloc(1024)         # => <Snow::Memory:...>
# Or, for convenience:
ptr = Snow::Memory[1024]                # => <Snow::Memory:...>

There are two important points when working with raw memory in snow-data vs. Fiddle:

First, the Memory object you allocate is garbage-collected when you allocate it using malloc. That memory will be freed when the Memory object is collected. This memory is managed by snow-data.

If you wrap a pointer using Snow::Memory.new or Snow::Memory.wrap, then the wrapped address cannot be freed. Wrapping an existing pointer means that memory is not managed by snow-data.

Second, the Memory object itself provides a large number of methods for working with numeric types and, if you define them, structural types as well.

This is to make it easier to construct data in memory without too much effort. First, for comparison's sake, we'll create a block of memory in Fiddle that encodes a float:

YOUR_FLOAT = 1.23456789
packed_float = [YOUR_FLOAT].pack('E')

# Pretend ptr is a Fiddle::Pointer here
ptr = Fiddle::Pointer.malloc(packed_float.bytesize)
ptr[0, packed_float.bytesize + 1] = packed_float

You don't really want to write a lot of code like that. In fact, you don't even really want to write functions to wrap code like that, because although packing and unpacking is reasonably fast for general usage, doing this at 60 frames per second isn't reasonable. Instead, I should really just go the direct route. Back in snow-data:

YOUR_FLOAT = 9.87654321

ptr = Snow::Memory[8]
ptr.set_double(0, YOUR_FLOAT) # put YOUR_FLOAT at offset 0 as a double
puts ptr.get_double(0)        # dereference a double at offset 0

# > 9.87654321

This avoids the need to pack and unpack data as strings, meaning we don't need to duplicate the data going into memory. This also avoids any indirection about which types are being accessed if using memory without defined structure.

In the above example, the memory is garbage collected. However, if you want to free it sooner, you can call the free! method:

# Because the other code might have the pointer around still, it can
# check if the pointer's been nullified:
ptr.null?                               # => false

# Free the pointer and nullify the address held by the Memory object:
ptr.free!

Finally, to avoid use-after-free errors, it is illegal to modify or double-free a piece of memory with snow-data:

# The object's still, but at least you know if the pointer's null:
ptr.null?                               # => true

# Double-free results in an exception:
ptr.free!                               # => RuntimeError: Double-free on Snow::Memory

That covers snow-data briefly without getting into more advanced features, like defining C structures. For more information, its README contains an example and the documentation may be helpful.

OpenGL in Ruby

This pertains specifically to opengl-core, again, so what I say here may not apply to other OpenGL bindings. If you're using those bindings, you're probably in the wrong place.

OpenGL core profile APIs assume you're going to take some piece of memory and load it into buffers, texture objects, and so on. In addition, you'll be responsible for allocating memory for some GL objects.

For this entire section, I'm referring mainly to OpenGL objects, such as textures and buffers. This isn't to be confused with Ruby objects, which are much different.

Older versions of opengl-core provided an auxiliary library to make it a little easier to work with some of these OpenGL objects. Since then, these have been moved to the opengl-aux gem. This document doesn't cover those.

To start, we'll allocate a buffer using both a String as a buffer and snow-data:

require 'opengl-core' # This require is implied from here on

# NOTE: GL commands won't work without a context, so if you need a playground
# running code snippets, you can use this gltest.rb gist:
#
#   https://gist.github.com/nilium/8949326

# Create a buffer object and store its name as a long in buffer0:
buffer0 = "0000"
glGenBuffers(1, buffer0)
buffer0 = *buffer0.unpack('L')

# Create a buffer object and store its name in the memory of buffer1:
buffer1 = Snow::Memory.malloc(4)
# Note the use of .address here -- it's needed to give GL the pointer.
glGenBuffers(1, buffer1.address)
# You could just as easily let the GC collect this, but I'll free it one go
buffer1 = buffer1.get_uint32_t(0).tap { buffer1.free! }

Both of these work, though I prefer the latter because it avoids an unpack, array allocation, and string parsing (and I'm biased in favor of snow-data).

Working with snow-data is a little verbose like this, though. We can clean it up a little bit using a Snow::CStruct:

# Define a GLObject type with a single 32-bit field: name.
GLObject = Snow::CStruct.new do
  uint32_t :name
end

# Create a new GLObject and store a buffer name in it:
buffer = GLObject.new                   # => <GLObject:...>
glGenBuffers(1, buffer.address)

# Print the buffer name (since we're not doing anything else with it here)
puts buffer.name                        # > 1

# Later, when we're done with it, free the buffer:
glDeleteBuffers(1, buffer.address)

# And optionally free the memory now that it's unused:
buffer.free!

And that's how we can define a new memory layout, create an object from it, and store a buffer object's name in it. If we want to create many of these, we can do something similar:

# Allocate an array of GLObjects
buffers = GLObject[12]                  # => <GLObject::Array:...>
glGenBuffers(buffers.length, buffers.address)

# Get a specific buffer
buffer_three = buffers[2]               # => <GLObject:...>
buffer_three.name                       # => 3

# Iterate over all buffers
buffers.each do |buf|
    glBindBuffer(GL_ARRAY_BUFFER, buf.name)
    # ...
end

# Free an array of buffers
glDeleteBuffers(buffers.length, buffers.address)

# And optionally free the memory
buffers.free!

This makes it a little easier to work with and reason about memory when using OpenGL, at least for me. The important point is it's easy to structure data in a way that reflects the types in use, instead of keeping track of different pack/unpack strings or calling get_{type} and set_{type} with the right memory offsets, especially when using arrays and structs with special alignments.

In addition, for vertex data, matrices, and so on, you can also use snow-math, a gem intended to handle 3D math in OpenGL efficiently. Like snow-data, it is implemented primarily in C, and has support for working with its underlying memory for use in OpenGL.

And that's the end of this document for now. If you have questions, creating an issue on one of the above projects is probably the fastest way to get help.