CMSC Final Old Material Test Quizzes with
complete solutions.
serial or sequential computing
doing a task in a sequence on 1 processes
paraellel computing
breaking up a task into subtasks and doing them concurrently on a set of processes
parallel architecture
set of nodes or processing connected by network
interconnection neworks
different topologies of connecting nodes together
system software
parallel programming model
core
single execution unit that has a private LI cache and can execute instructions independentley
processor
several cores on a single integrated circuit or chip are multi core processors
socket
physical connection into which chip or processor is inserted
node
,packaging of sockets
job scheduling
each user submits parallel program for execution to job scheduler and that decides what job to
schedule next and how many nodes to allocate
compute nodes
dedicate nodes for running each job (only accessed when allocated to the user)
login nodes
shared by all users to compile programs, submit, etc
cluster
cluster of nodes, typically put together with commodity hardware
thread
thread/path of execution managed by OS (share same memory address space)
process
heavy-weight, don't share resources such as memory, file descriptor, etc
strong scaling
fixed total problem size but run on more processes
ex. sort n numbers on 1, 2, 4 ... processes
weak scaling
,fixed problem size per process by increasing total problem size
ex. n numbers on 1 p, 2n on 2p ... processes
parallel speedup ratio
time on 1 process / time on p processes
parallel efficiency
speedup / process
Amdahl's Law
speedup is limited by the serial portion of the code (serial bottleneck that can't be parallelized)
Amdhal's Law Formula
1 / [ (1 - f) + (f / p) ]
where f is the fraction of program that can parallelized
intranode communication
communication among cores within a node
internode communication
communication among cores on different nodes connected by a network
bulk synchronization programs
all processes compute simultaneously then synch. together
, Single Instruction Single Data (SISD)
A uniprocessor
data --> processeor <--- instructions
Single Instruction Multiple Data (SIMD)
Same operation is carried out on multiple pieces of data at once
Multiple Instruction Multiple Data (MIMD)
Different instructions carried out on different data at the same time
single instruction multiple threads
threads execute lock in step (GPUs)
Single Program Multiple Data (SPMD)
all processes execute the same program/instructions but act on different data -- MIMD
parallelization
all processes start from the same executable, each process has a unique rank, processes mak take
different branches and operate on different data based on their rank
steps to writing a parallel program
1. find serial algo
2. how to distribute data among threads/processese while minimizing data movement
complete solutions.
serial or sequential computing
doing a task in a sequence on 1 processes
paraellel computing
breaking up a task into subtasks and doing them concurrently on a set of processes
parallel architecture
set of nodes or processing connected by network
interconnection neworks
different topologies of connecting nodes together
system software
parallel programming model
core
single execution unit that has a private LI cache and can execute instructions independentley
processor
several cores on a single integrated circuit or chip are multi core processors
socket
physical connection into which chip or processor is inserted
node
,packaging of sockets
job scheduling
each user submits parallel program for execution to job scheduler and that decides what job to
schedule next and how many nodes to allocate
compute nodes
dedicate nodes for running each job (only accessed when allocated to the user)
login nodes
shared by all users to compile programs, submit, etc
cluster
cluster of nodes, typically put together with commodity hardware
thread
thread/path of execution managed by OS (share same memory address space)
process
heavy-weight, don't share resources such as memory, file descriptor, etc
strong scaling
fixed total problem size but run on more processes
ex. sort n numbers on 1, 2, 4 ... processes
weak scaling
,fixed problem size per process by increasing total problem size
ex. n numbers on 1 p, 2n on 2p ... processes
parallel speedup ratio
time on 1 process / time on p processes
parallel efficiency
speedup / process
Amdahl's Law
speedup is limited by the serial portion of the code (serial bottleneck that can't be parallelized)
Amdhal's Law Formula
1 / [ (1 - f) + (f / p) ]
where f is the fraction of program that can parallelized
intranode communication
communication among cores within a node
internode communication
communication among cores on different nodes connected by a network
bulk synchronization programs
all processes compute simultaneously then synch. together
, Single Instruction Single Data (SISD)
A uniprocessor
data --> processeor <--- instructions
Single Instruction Multiple Data (SIMD)
Same operation is carried out on multiple pieces of data at once
Multiple Instruction Multiple Data (MIMD)
Different instructions carried out on different data at the same time
single instruction multiple threads
threads execute lock in step (GPUs)
Single Program Multiple Data (SPMD)
all processes execute the same program/instructions but act on different data -- MIMD
parallelization
all processes start from the same executable, each process has a unique rank, processes mak take
different branches and operate on different data based on their rank
steps to writing a parallel program
1. find serial algo
2. how to distribute data among threads/processese while minimizing data movement
