I want to share my elaboration about Parallel Matrix Multiplication I have writed (as extra task) by the way Parallel Computations which is one of my subject in Silesian University of Technology in the Computer Science course I attend. The elaboration have been written of course in Polish.
I have sloved this problem very simply in ADA95 programming language, which gives an opportunity to write parallel programs very easly in Pascal-like syntax.
My solution of this problem divides an elementary operations between 1 to N provided processors, distributed with shared memory CRCW PRAM model (Parallel Concurrent Read and Concurrent Write). In particular, you can fire up computing in only one processor. Elaborate discuss about problems with value of memory cells working as an accumulator in adding (+) process. It’s overall only.
- Algorytm skalowalnego zrównoleglonego mnożenia macierzy (PDF, ~0.75 MB)
- MatrixParallelMultiplication.adb (ADA95 implementation)
Finally, I have compiled the code under GNAT 2011 compiler in GPS 2011 IDE (GNAT Developer Studio). For people interested in ADA95 programming and Parallel Computations I recommend some polish books avaiable for example in Silesian Libraries:
- Porębski, W. (2003). ADA95 Wprowadzenie do Programowania. Michałowice: Komputerowa Oficyna Wydawnicza “HELP”, Piotr Gomoliński.
- Czech, Z. (2010). Wprowadzenie do obliczeń równoległych. Warszawa: Wydawnictwo Naukowe PWN.
- Ben-Ari, M. (2009). Podstawy Programowania współbieżnego i rozproszonego (strony 99-133). Warszawa: WNT.
Xin Wang reveals another ways to slove this problem in Scalable Parallel Matrix Multiplication Algorithms with
Application to a Maximum Entropy Problem publication, but it is only math theory, without implementation.
You can easly write parallel prorgams using MPI library for C++ or Parallel library Java. In general, there are many ways to implement algorythms, but the most important in Parallel Computations is not implementations, but problem decomposition and hardware structure of computing processors which exchanges information with each other.