Archive for the ‘Uncategorized’ Category.

It’s done!

Today I finally finished my Ph.D. My defence was on Monday 22, and after some minor corrections, I submitted the final version. It bounced once because of a formatting issue, but today the library approved it and it’s all over.

I was here for 4 years and 8 months. I’m taking many good memories with me, it was certainly a good time! All the conference trips, the freedom to work or learn whatever you want, were without doubt the best of the “professional” live as a Ph.D. student. Also being able to manage your own time was a big big plus. You could spend time reading, learning stuff not completely related to your thesis, enjoy a bit of curiosity over different topics or fields.

I’m leaving Canada in a couple of days. Not for long though, since I won’t miss my convocation ceremony in mid June.

I’m going back to Chile to work in Akori. A company I founded together with an old friend from undergrad. It’s going to be an exciting and uncertain new experience. At times I’ll probably miss the flexibility provided by the academia though :-).

If you are curious about the thesis itself, it’s already available! Download it here!

Increasing the memory limit in Go

Yeah, it’s been a while, but instead of talking about why I haven’t written anything, I’ll just jump into the good stuff.

I’ve been developing in Go lately. If you haven’t tried it yet, I totally recommend it. Honestly, when you first look at the language, you tend to think that there is nothing new in there. That may be true. Many of the features are not unique, and you can find them in other languages, yet the combination is unique. After one week of using Go I didn’t want to move back to C++ (what I usually use). There are a couple of things I miss from time to time, but the gain is so big that I don’t mind.

The problem. I’ve had a couple of problems dealing with the kind of software I usually implement. Today I found one that did worry me a lot, I can’t use more than 16GB of RAM! After that, my code will just crash with an “out of memory” error. In general this is not too bad, but for the particular algorithm I’m running, I need more than 16GB. Luckily, I found out that the limitation is temporal and that you can lift it up by changing just a couple of lines in Go’s source code. So here it goes.

I will assume you installed Go in $HOME/bin/go (if not, don’t worry, we are going to re-install it, right there).

The first step is to get the source for Go:

$ wget

Once that finishes, decompress the package:

$ tar xvfz go1.0.2.src.tar.gz

Now, using your favourite editor, change the following files as explained here:

  • In file go/src/pkg/runtime/malloc.h
    Change line 119, where it says “MHeapMap_Bits = 22,” for “MHeapMap_Bits = 25,”
  • In file go/src/pkg/runtime/malloc.goc
    Change line 309, where it says “arena_size = 16LL<<30;” for “arena_size = 128LL<<30;”

And that’s it, now lets install this.

$ cd go/src
$ ./all.bash
$ cd ../..
$ mkdir -p $HOME/bin/go
$ mv go $HOME/bin/

Now we are ready. Remember to set the environment variable in your .bashrc:

export GOROOT=$HOME/bin/go
export PATH=$PATH:$GOROOT/bin

You are ready to enjoy 128GB. If you want 64GB, change the 25 for 24, and the 128 by 64.

This is supposed to be experimental, so please don’t blame me if things crash. There has to be a reason for that limitation to be there. In my case, I was able to get away with this and run my code. In this particular case is the (expensive) construction of a compressed text index, so the thing runs, outputs, and that’s it. Afterwards I process the output with other binaries. I’m just happy if the program manages to go through the construction, without much care on the resources I’m using at the time.