Introduction To Parallel Computing!

Meet Prof X and Student Y

Student Y keeps asking all the “why” questions and answers to these by our very own Prof X will probably help you solve your doubts and clarify your concepts!

Ahh! So now that you’ve met the two protagonists, what are we waiting for?

Prof X : So a long time ago, our hero of the story Mr. Comp was a very hardworking guy. His services were quite expensive and were available only to the elite and his special resources were used by only knowledgable people equipped with special skill sets just to exploit the services of Comp in the best possible manner to make every buck worth it.

Student Y : Looks like he was a big shot! By the way, I do get the hidden implicit references you are making professor. Yeah I know that in olden days computers were a big expensive things and the cost of running them used to run into millions.  Hence, only expert programmers were allowed to handle. With computing resources becoming cheaper day by day, it is now accessible to millions around the world.

Prof X : Haha! I give it to you. Yes, the technological advancements have been very quick but do you know that these advancements actually happen to stick to a pattern. To put it in a dramatic way, the prophecy (Moore’s law) of Robert Moore continues to be fulfilled. Now that the capabilities of our computers continue to grow, we should also learn to harness these powers. This is what parallel computing is about.

Student : Yeah true. I read it somewhere that the iPhone itself is almost 3 times powerful than the supercomputers in the 1980’s but what’s there for me to learn about all of this on the software aspect of it? Like I mean that it’s like Mr Comp has grown more muscle and he still does the same task but probably now with more speed and power.

Prof X : Haha no! There is a vast ocean of knowledge regarding this new era of computing. Coming back to our story. Mr Comp did actually start becoming more strong and powerful but how much stress could he even take? Bad for him. Right? So after some time he evolved himself into a company of more than 1 employee. See now where do we come in the picture?

Student : Huh? I’m not totally not getting you Prof X.

Prof X : Wait for it.  He then started out by created an excellent team of professionals. To handle the tasks that were being given to the Memory department in a faster manner, we recommended him to hire L1, L2, L3 and a few others and also at the crux of the company he added a few more people capable just like him.

Student : Haha! I get it now. It’s like a multi processor machine having a cache structure and hence those names L1, L2 and L3. It still doesn’t answer my previous question professor. More people more work done. How are things changing for us?

Prof X : Smart. Yes to some extent Comp was able to your same old job in a quicker way. Although the labour had become cheap, we were still sticking to our old ways. The responsibility fell on us to best use Mr Comp’s resources. We know have to think in a way how best to use those additional resources now available to us. We must be able to think what task would each of them do? How to make sure that there are no clash of interest among them? We would be giving orders to them and just make use of the hierarchical and structured resource provided to us.

Student : Oh! Now I’m getting it. It’s the developers onus to make best use of this new computer architecture and that is what parallel computing and high performance computing is about. Right?

Prof X : Yes, you’re indeed right. To harness these resources best you need to understand and learn a little of hardware aspects as well as be able to direct the workers of Mr Comp in a more efficient manner (software). We must be in a situation to be best able to exploit the resources of Mr Comp and finish our tasks early so that we can go home and have some good family time.

Student : Thanks Prof X!! You certainly have incited a spark in me but it does take some effort to make that spark turn into a fire for me to turn *dangerous*. I will do a bit more reading and come back to you.

Prof X : Before you leave, I think I should introduce you to a person from the open source world who is here to help us for free. With his help you’ll be better able to make best use of Mr Comp’s resources. Meet none other than Mr. OpenMP. He is like a middle man who makes our life simpler.

Student : Thank you professor! Before leaving, could you suggest me a few books or good sources of information?

Prof X :  There happen to be a lot of books out there. The few ones that I’ve been able to land my eyes upon and find it interesting are –
1. Parallel Programming in C with MPI and OpenMP, Michael Quinn
2. Using Open MP, Barbara Chapman , Gabrielle Jost, Ruud Van Der Pas 
I’ll update the list whenever I find something new and interesting. These books will also discuss also about other techniques. So do read upon them to. Open MP documentation from LLNL also happens to be good source to read from.

Student : Thanks professor!

Prof X : Anyways, go into the wild. Google a bit, read a bit, experiment a bit and come back with lots of queries and doubts!

I guess a few of your “why” questions were answered and hopefully Prof X and his conversation with Student Y turned out to be helpful for you.

Thank you.

Signing Off,
Sumedh Arani.