This book is a systematic and comprehensible guide that introduces the reader to the dynamically evolving world of parallel processing. It is ideal for students in the field, researchers, and programmers, with the only prerequisite being some basic knowledge of algorithms and programming.

The book examines the design of parallel systems, both at the level of computer architecture and at the level of programming. Key concepts such as multi-computers and multi-processors are explained, the main performance evaluation metrics of parallel algorithms are presented, and key architectures of parallel processing networks are described.

Additionally, extensive reference is made to the implementation of parallel algorithms on shared-memory parallel processing architectures, such as graphics cards (GPU) through CUDA standards and the OpenCL language.

Contents:

• Parallel processing architectures
• Interconnection networks
• General issues of parallelization
• Task parallelization
• Nesting loops
• Dependencies in nested loops
• Scheduling
• Visualization
• Computations on the GPU: Parallel processing on graphics cards
• Mathematical and algorithmic tools
• Visualization of nested loops
• Systolic arrays of processors