Uniform Memory Access
Non-Uniform Memory Access
Cache-only Memory Access
Hybrid (Distributed-Shared Memory)
Levels of parallelisation:
Splitting the instruction into multiple stages. allowing multiple instructions to occupy different stages in the same clock cycle.
Allowing multiple instruction to pass through the same stage. Scheduling how instructions are executed together.
breaking a task into sub components
of tasks to processes/threads. 3. Mapping
of processes to cores
A process is a program in execution.
A process is broken into several components.
Process are abstraction that let you exploit multiple cores/parallel processes.
Processes were initially created to allow for multitasking.
There are two types of multitasking.
Same operation applied to different elements. Operations are independent
Partition the data to solve problem.
Exploited by SIMD computers extensively. (Also SPMD).
Partition the task to solve the problem.
Analysed using a task dependence graph. Form direct acyclic graph, which represents the control depdendency between tasks.
Critical path length: fastest completion time
Degree of concurrency = Total work/ critical path length/
Task T creates a number of child tasks. Tasks then work in parallel to excecute the program. After that, a join is used.
Specifies function call to be executed.
Same program executed on different processors but operate on different data. Single instructions executed synchronously by the different threads on different data.
Master assigns tasks to slaves.
Server computes requests from multiple client tasks concurrently.
Passing data between components of a pipeline.
Data structure that threads can access to retrieve tasks for execution.
Producer produces data into a common buffer which are then consumed by the consumers.