CSE 8163

PARALLEL AND DISTRIBUTED SCIENTIFIC COMPUTING

  Prof. Ioana Banicescu.

 HOURS:

CLASS: Mon/Wed 2:00-3:15, Butler 102.

OFFICE: 320 Butler Hall, Monday 3:30-5:30, Wednesday 3:30-4:00. Other times by appointment only. Email:ioana@cse.msstate.edu.

 325-2756.

RECOMMENDED TEXT BOOK AND READINGS FROM:

Ananth Grama, George Karypis, Vipin Kumar, and Anshul Gupta Introduction to Parallel Computing (2nd Edition), Addison-Wesley, 2003

Rob Bisseling, Parallel Scientific Computations - A structured approach using BSP and MPI, Oxford University Press, 2004

Afonso Ferreira and Jose D. P. Rolim, Parallel Algorithms for Irregular Problems: State of the Art, Kluwer Academic Publishers, 1995

C. Xavier and S. S. Iyengar, Introduction to Parallel Algorithms, John Wiley & Sons, 1998

Additional readings from research papers will be indicated in class.

PREREQUISITES:

CS4163/6163 Designing Parallel Algorithms course in high performance computing is a must. Two courses in high performance computing, and/or additional courses in algorithms, compilers, operating systems, computer architecture, and experience with message passing interface and programming with threads, are preferred.

OBJECTIVES:

This course evolves out of the Designing Parallel Algorithms and a series of other high performance computing courses and related topics taught at the Mississippi State University. It is targeted towards graduate students in sciences and engineering who are interested in solving computationally intensive problems on parallel and distributed machines. The class will survey recent advances in parallel algorithms for scientific computing and will aim to discover their impact on improving performance of scientific applications. Since students come from different departments and backgrounds, the lectures will attempt to emphasize an even spread of topics in algorithm design, numerical methods, computer architecture, software and runtime environments. Students will spend time on particular parallel algorithms for scientific problems and their implementations on parallel and distributed machines. There might be a few invited talks given by recognized researchers (i.e., affiliated with the MSU research centers such as the ERC, or other institutions). The invited talks will be related to recent research advances in preferred topics related to numerical analysis, scientific applications, runtime environments, performance analysis, and others.

COURSE REQUIREMENTS:

The class will study a variety of algorithms, their applications, and tradeoffs between different solutions. Issues are performance analysis, evaluation and prediction will be emphasized. Recent advances in task scheduling and load balancing algorithms will also be introduced and discussed in detail. There will also be discussions on issues such as parallel computer architectures (memory hierarchy, interconnection networks, latency and bandwidth, parallel I/O), and software systems, with the aim of understanding their capabilities, costs and limitations. Students will make use of recent technology through a number of software packages and programming environments appropriate to the topics addressed. These will be used in comparing the competitiveness and evaluating the performance of different implementations through a variety of criteria. Students will draw conclusions regarding preferred algorithms, methods, programming paradigms, and programming environments and tools for parallel and distributed computing. Among sample problems used in class, the following case studies could be enumerated: linear algebra and large systems of differential equations arising in global climate modeling, finite element, finite volume methods, and hierarchical methods for particle simulations in computational field simulation (i.e., computational and molecular modeling, computational fluid dynamics, plasma physics).

GRADING: Assignments 20%, Presentations 10%, Midterm/Papers/Projects 30%, Final 40%.

Attendance: more than three absences will influence grades at border line.

 ANNOUNCEMENTS:

Students are responsible for checking the announcements regularly. For important announcements, click here.

COURSE OUTLINE:

The course will cover material from the following topics:

Fundamentals: Introduction to Scientific Computing.

scientific problems and their characteristics

the impact of parallel programming on performance and numerical properties

Fundamentals of parallel and distributed computing: theory and practice

models of parallel computation (i.e., PRAM, BSP, LOGP, etc)

parallel programming paradigms (data parallel, message passing and shared memory with multithreading)

mapping algorithms onto architectures (partitioning, allocation, scheduling)

performance analysis, evaluation and prediction (load balancing, communication, overhead)

A closer look at numerical properties, parallel algorithms and architectures: methods and tradeoffs

The impact of numerical properties on the performance of scientific applications cases studied (i.e., discrete Poisson equation using Jacobi or Multigrid methods, SOR, Conjugate Gradient and the FFT; hierarchical methods for the N-Body problem, etc)

Algorithms of common use (i.e., parallel sorting, parallel matrix multiplication, graph partitioning, fast algorithms using trees)

Recent technological advances in parallel architectures and their impact in the programming design methodology (i.e., cluster computing, parallelizing compilers, etc).

Scheduling and load balancing algorithms: theory and practice

Advanced scheduling methods, dynamic task scheduling, and loop scheduling

Load balancing methods (deterministic and stochastic algorithms, optimality analysis, methods using combinatorial optimization)

Important issues visited in case studies

Recent progress in high accuracy and high performance of linear algebra algorithms

Sources of parallelism and locality in simulation

The impact of memory hierarchy in the algorithm design

Scalability analysis of parallel applications

Combinatorial aspects in scientific computing

The impact of new technology in the design of parallel scientific applications

ACADEMIC HONESTY:

Students are expected to follow the CS department's policy on academic honesty with regards to homework, projects and exams.