Raspberry Pi 3 Cluster Computer - MPICH2 & MPI4Py

Показать описание

Demonstrates how to build a cost effective cluster computer using Raspberry Pi 3's, and how to install the software to run MPI applications. This is a three node system, but can easily be scaled to larger sizes if so desired.

Raspberry Pi DIN Rail Mount:

WAGO 24 VDC P/S 787-1602
WAGO 5-port Giga Switch 852-1111

Рекомендации по теме

Комментарии

Thank you so much for making this video. I decided to do everything on arch linux, and it has been quite difficult trying to translate. But you made the process quite simple. I still have a fourth node that will not respond. But I did manage to get three working. Plus, mine are pi 2's. Once again, thank you for making such a resourceful video. I now understand. MPI for pi!

mrmrapparatus

can you make a forum post with all the steps so i can read all the commands properly?

Excellent video - I'm glad to see that you used a Network File System. I saw a video where someone copied each file to each server. Great job!

FixWPMe

excellent video. much easier to understand than the docs I've seen online since I started using zeros.

RossPotts

I had something similar in mind for my own numerical simulations or BOINC stuff.

But before spending hundreds of Euros for RPi3 boards, I did some tests with a special version of my own nbody code which is written in C, highly optimized and utilizes the OpenMP library for parallel computing. My benchmark focussed on floating-point performance with negligible RAM consumption. What are my results? Well, it's disillusioning. My benchmark task (compiled with gcc) took 65 minutes on a single core of my RPi3, while a single Core i5-6500 (gcc again) solved the problem in 65 seconds! Using four threads, the RPi3 still took more than 18 minutes, while my Intel Core i5-6500 got that job done in 18 seconds. The Intel C Compiler makes things even more ridiculous: just 11 seconds @ four threads.

adavistheravyn

very cool video! I did this with a 6 Pi-3 cluster and the recent versions of python. One recommendation, if you install pdsh on the pi's and run from a desktop you can save yourself a lot of typing by issuing commands in parallel to the pi's. That will definitely keep your sanity if you are building larger clusters. I saw a "WHY?" in the comments, I am using the pi's to simulate web traffic for load testing.

tomknorr

Great easy to understand content for us noobs, thanks Kurt.

tomjohnson

All you static IP stuff is not needed if you set each to have a unique DNS name. The default is raspberrypi.local and you do each one individually as you're imaging the next. Also, Jessie-Lite would allow for smaller base image and would save you a lot of SD space considering you're using 8gig cards.

MatthewPotter

Short headsup
If you're looking to do this tutorial with a modern linux system; ifconfig has been replaced in nearly every distro by the iproute2 package.
The short of it is that you now use the " ip" command.
ip [OBJECT] [COMMAND]
Is the standard format so use "ip address show" to show your Pi's IP address.

Also it's generally considered bad practice to use the /etc/fstab file. One problem is that your device will refuse to boot up if it can't find every single little thing in the file. So got network issues or node1 died? Literally can't use the others until you fix or replace node1, or clear the SD card and write a new image to it.
Instead use systemd and a .mount file in /etc/systemd/system/ it'll take some getting used to, but once it works it's a lot more robust.

tiaxanderson

Do you really need to make the ip addresses static? Other tutorials seem not to do it.

dhmketu

Wonderfully informational video Kurt.
I'm curious though, can this be done with Java rather than Python? While I intend to add Python to my list of languages I have an understanding about, I'd like to do this with Java first.
Also, could one use an external HDD/SSD for the shared folder location?

SorenEragon

How effective would a cluster of Raspberry Pi's running as Beowulf Linux distributed cpu farm be? Do the cpu's have enough horsepower that a meaningful Beowulf cluster could be created? = budget supercomputer. I genuinely appreciate anyones feedback on this as an idea. Possibly 1 regular pc master, and a whole bunch of raspberry pi nodes.

dhewton

What usages does a Cluster PC have i am curious and unknowledged :C

aeozFX

The question is not how to build it but twhat to do with it.

llothar

what does it mean by clustering raspi? i mean what does it mean for the GPIO? and what if i set the GPU to 128, i need it to run image processing + sensors

briwicklab