Attempt to build a Spark cluster on Raspberry Pi

Fig. 1. An Artik10 board

We started off building a Spark cluster using 10 Raspberry Pi1 model B+ since that is what was available in the lab at that point. Just to inform, a Pi1 model B+ has 512M of memory, ARMv6 single core, and 700 MHz clock speed. We made one of these Pis as the Spark master and the rest as workers. Spark version v2.1.0-rc1 was used.

The initial tests included simple word count and sorting examples. We expected it to run slow because of memory constraints and that is what happened. Running word count and sorting algorithms on a text file of less than 10 words took like an hour. More complicated algorithms never ran to completion. We decided to halt our testing and order Pi3 models. Along with Pi3 models, we also ordered 5 x Artik10 boards. Each Artik10 board has 2G memory, 8 threads, 1.3GHz, and a ARMv7 Processor rev 3. Compared to this, a Pi3 has 1G memory, 4 threads, 1.2GHz, and a ARMv7 Processor rev 4 (v7l).

I am not diving into more technical, low level details on how we setup a Spark cluster but if you have specific questions, please leave comments. I would gladly help.