2021 High Performance Computing Lecture 0 Prologue Part1 💻

Lecture 0 - Prologue 🖥️ - Part One

Advanced Scientific Computing
16 university lectures with additional practical lectures for hands-on exercises in context
University of Iceland, School of Engineering and Natural Sciences
Faculty of Industrial Engineering, Mechanical Engineering and Computer Science

Spring 2021

Course Outline
1. High Performance Computing
2. Parallel Programming with MPI
3. Parallelization Fundamentals
4. Advanced MPI Techniques
5. Parallel Algorithms & Data Structures
6. Parallel Programming with OpenMP
7. Hybrid Programming & Patterns
8. Debugging & Profiling & Performance Analysis
9. Accelerators & Graphical Processing Units
10. Parallel & Scalable Machine & Deep Learning
11. HPC in Health & Neurosciences
12. Computational Fluid Dynamics & Finite Elements
13. Systems Biology & Bioinformatics
14. Molecular Systems & Material Sciences
15. Terrestrial Systems & Climate
16. Epilogue

[mkhadka]

Thank you for providing this excellent HPC course as a free resource. Extremely Grateful!

[hoaxuan]

You can use use a simple evolution algorithm, Continuous Gray Code Optimization, to train neural networks quite effectively. Each device (GPU CPU or whatever) gets the full neural and part of the training set (which can be local and private.) The same short sparse list of mutations is sent to each device which returns the cost for its part of the training set. The costs are summed and if an improvement an accept message is sent to each device else a reject message. Not much data moves around per unit time. There are no problems with gradients etc and in many cases the devices can be anywhere on the internet, unless each device has a very small fragment of the training data.

[lucam]

thank you for uploading the lectures. very helpful for beginners.

[Poeterish]

Thank you so much for these fantastic (and free) resources!

[-MinutesAcrosstheWorld]

Thanks so much for the lecture - One of the best courses. Very helpful!

[vincenthus]

Thanks a lot for sharing this course! This really helps the world :)

[vpanagiotou]

Sir could you please upload the video recording for lectures 12-16 ??
I am studying computer engineering and informatics, and i want to expertise in bioniformatics, so these lectures would be a great opportunity to broaden my knowledge in this field.
Your lectures are extemely helpful and are a great paradigm for all university professors

[kanglei]

Hello professor, where is the actual code of the project?

[dylanvernon]

Are the video recording for lectures 12-16 available on YouTube or on some other platform?

[zhaomonto]

Are the video recording for lectures 12-16 available on YouTube or on some other platform?