Course content

Introduction to large and distributed data volumes. Introduction to distributed techniques. How to design data-intensive applications? Reliability, scalability, and maintainability; how we need to think about them; and how we can achieve them. Data models and query languages. Indexing and storage techniques. Encoding of data. Replication, partitioning and transactions. Fault models, consistency and consensus.

Learning outcome

Learning outcome

Knowledge:

By completion of this course, the candidate should be able to explain

1. reliable, scalable, and maintainable distributed systems

2. data models and query languages- indexing- and data storage methods

3. formats for encoding of data

4. models of replication

5. models of partitioning

6. theory of transactions and concurrency

7. fault models

8. consistency and consensus

9. algorithms for consensus

10. synchronization of clocks

11. distributed debugging

12. examples of distributed databases: Amazon Dynamo and Google Spanner

Skills:

By completion of this course, the candidate should be able to

1. develop applications with big data using standard database products.

2. evaluate existing systems and solutions for distributed storage and management of data

3. combine tools to build the properties you need

4. develop new systems for distributed storing and management of data

General competence:

By completion of this course, the student should be able to explain distributed systems.

Learning methods and activities

Lectures, exercises, projects and self-study.

There are compulsory exercises in the subject.

There are two projects involving programming with big data volumes and which are done in small groups.

Compulsory assignments

• Exercises

Recommended previous knowledge

TDT4120 Algorithms and Data Structures and TDT4145 Data Modelling and Database Systems, or equivalent.

Course materials

Information given at start of term.

Credit reductions