Systeemontwerp samenvatting
Inhoudsopgave
Hoofdstuk 0: Introduction .............................................................................................................................. 6
Systeemcomponenten ........................................................................................................................................ 7
Systeemontwerp in bedrijven en deployen van een systeem ............................................................................. 7
Systeemarchitectuur .......................................................................................................................................... 8
Systeemrequirements......................................................................................................................................... 8
4 + 1 view model van softwarearchitectuur ....................................................................................................... 8
Reactive Manifesto .......................................................................................................................................... 11
Hoofdstuk 1: Microservices .......................................................................................................................... 13
Architectural style ............................................................................................................................................ 13
Wat is architecturale stijl ................................................................................................................................. 13
4 Architecturale stijlen ..................................................................................................................................... 13
Microservices ................................................................................................................................................... 16
Wat is een service ............................................................................................................................................ 16
Wat is een microservice ................................................................................................................................... 16
What is loosely coupled? .................................................................................................................................. 17
Nadelen/Challenges van het MicroService pattern .......................................................................................... 17
Decompositie.................................................................................................................................................... 18
Stap 1: identificeren en begrijpen van systeemoperaties................................................................................. 18
Stap 2: Identificeren van services door business capabilities ........................................................................... 19
Stap 3: Definiëren van service API’s en collaborations ..................................................................................... 20
Finaliseren van de services ............................................................................................................................... 21
Problemen bij decompositie ............................................................................................................................. 21
Interactiestijlen tussen micro-services ............................................................................................................. 22
Interactie is een logisch en technisch probleem ............................................................................................... 22
Interactiestijlen ................................................................................................................................................ 22
Synchrone communicatie: gRPC ....................................................................................................................... 24
Handling partial failure .................................................................................................................................... 24
Bulkhead: limiet zetten op het aantal uitgaande requests .............................................................................. 25
Circuit breaker .................................................................................................................................................. 25
Fallback strategy to recover from failing call ................................................................................................... 25
Asynchrone communicatie: Messaging ............................................................................................................ 25
Messaging supports all interaction styles ........................................................................................................ 26
Specification for messaging-based service API ................................................................................................ 26
1
, Message broker technologieën ........................................................................................................................ 27
Design issues: competing consumers ............................................................................................................... 27
Hoe lost Kafka die problemen op? ................................................................................................................... 27
Kafka at work: voorbeeld MMOG .................................................................................................................... 28
Design issues: handling duplicate messages .................................................................................................... 29
Synchrone interactie reduceert beschikbaarheid vd applicatie ........................................................................ 30
Interactie is orthogonaal (niet gelijk aan) de gebruike communicatie............................................................. 30
Eliminating Synchronous interaction................................................................................................................ 31
Saga’s ............................................................................................................................................................... 32
Transactiegaranties in een monoliet (monolithic system) ............................................................................... 32
Distributed transactions: 2PC (2 phase commit) (bij microservices) ................................................................ 32
Saga: sequentie van lokale transacties ............................................................................................................ 33
Transactionele messaging ................................................................................................................................ 33
Compensating transactions ............................................................................................................................. 33
Voorbeeld: rejecting an order .......................................................................................................................... 34
Concurrent Saga’s ............................................................................................................................................ 34
Saga coordination: hoe wordt de saga uitgevoerd? ........................................................................................ 34
State Machine model ....................................................................................................................................... 35
Business logica van een enkele microservice ................................................................................................... 36
Business logic organization .............................................................................................................................. 36
Building blocks for a good domain model ........................................................................................................ 37
= Een klasse: heeft ............................................................................................................................................ 37
Putting everything in place .............................................................................................................................. 39
Example: restaurant order management ......................................................................................................... 39
Queries in a microservice architecture ............................................................................................................. 40
The API composition pattern ............................................................................................................................ 40
Design issue: wie speelt de rol van API composer? .......................................................................................... 40
Moeilijkheden bij queries naar 1 service .......................................................................................................... 41
Command-Query responsibility segregation: oplossing voor moeilijkheden ................................................... 41
Hoofdstuk 2: Deployment and orchestratie .................................................................................................. 43
From the bottom to the top: Development naar productie ............................................................................. 43
Making your service production ready ............................................................................................................. 44
Configureerbare services .................................................................................................................................. 44
Push-based externalized configuration ............................................................................................................ 44
Pull-based externalized configuration .............................................................................................................. 44
Observable services .......................................................................................................................................... 44
Log aggregation ............................................................................................................................................... 45
Distributed Tracing ........................................................................................................................................... 45
2
,Audit logging .................................................................................................................................................... 45
Application metrics .......................................................................................................................................... 46
Containers als een unit voor deployment ......................................................................................................... 46
Probleemstatement: development to operations ............................................................................................ 46
Virtual machine images als deployable artifacts ............................................................................................. 46
Host virtualisatie: VM’s en containers ............................................................................................................. 47
Linux containers technology stack ................................................................................................................... 47
Functies van Linux Kernel: Linux Control Groups (CGROUPS) ........................................................................... 47
Functie van de Linux Kernel: Linux namespaces ............................................................................................... 48
Functie van Linux Kernel: changing root .......................................................................................................... 48
Functie van Linux Kernel: Secure containers .................................................................................................... 48
Wat is Docker? ................................................................................................................................................. 48
Docker vs Linux containers ............................................................................................................................... 49
Docker architectuur .......................................................................................................................................... 49
Building images from Dockerfiles ..................................................................................................................... 49
Images layers ................................................................................................................................................... 50
Container vs image........................................................................................................................................... 50
Building an image ............................................................................................................................................ 50
Union file system .............................................................................................................................................. 50
Container Orchestration................................................................................................................................... 51
Wat doet een Orchestration framework? ........................................................................................................ 51
Kubernetes architectuur ................................................................................................................................... 51
Kubernetes concepten: PODS en deployment objecten ................................................................................... 52
Multicontainer pods ......................................................................................................................................... 52
Service discovery: the need of a service registry .............................................................................................. 52
Hoe gaan clients de services in het registry vinden? ........................................................................................ 53
Servcie Discovery in Kubernetes? ..................................................................................................................... 53
Load balancer ................................................................................................................................................... 53
Rolling updates: zero-downtime ...................................................................................................................... 54
Andere deployment operaties met een load balancer ..................................................................................... 54
Dimensions of cloud computing ....................................................................................................................... 55
Essentiële karakterisitieken .............................................................................................................................. 55
Stakeholders (service models) .......................................................................................................................... 55
De rol van de cloud provider ............................................................................................................................ 55
Multi-tenant homogeneous pooled resources ................................................................................................. 56
Cloud service models: X as a service (fill in the X) ............................................................................................ 56
Deployment models (nieuwe pijl, de pijl omhoog) ........................................................................................... 58
Elastic Auto-scaling .......................................................................................................................................... 59
3
, Cloud user objectives........................................................................................................................................ 59
Statisch vs elastisch schalen ............................................................................................................................. 59
Verschillende soorten workload ....................................................................................................................... 60
Types of scaling ................................................................................................................................................ 60
Verticaal schalen .............................................................................................................................................. 61
Horizontaal schalen.......................................................................................................................................... 61
Autoscaling....................................................................................................................................................... 61
Case study; auto-scaling in Amazon EC2 .......................................................................................................... 62
Standy pooling ................................................................................................................................................. 62
Scaling guidelines ............................................................................................................................................. 63
Multisite deployment ....................................................................................................................................... 63
Two stage load balancing ................................................................................................................................ 63
Example: Google Compute Engine ................................................................................................................... 64
Hoofdstuk 3: Data intensieve systemen: Deel 1 ............................................................................................... 66
Challenges of modern data .............................................................................................................................. 66
Data-intensieve systemen ................................................................................................................................ 66
Datamodellen................................................................................................................................................... 66
NoSQL ............................................................................................................................................................... 68
Data storage and retrieval ............................................................................................................................... 69
Twee soorten storage engines ......................................................................................................................... 69
Hoe storage engines beoordelen? .................................................................................................................... 69
B-tree index for page-based storage ................................................................................................................ 70
Log-structured storage engines ....................................................................................................................... 70
Betrouwbaarheid van een DB .......................................................................................................................... 71
Comparing B-trees to LSM-trees ...................................................................................................................... 71
Ster schema voor analyse................................................................................................................................. 72
Relationele data opgeslagen per kolom, niet per rij ........................................................................................ 72
Distributed data ............................................................................................................................................... 73
Waarom data distribueren? ............................................................................................................................. 73
Patterns voor datadistributie ........................................................................................................................... 73
Distribute data: replicatie ................................................................................................................................ 74
Replicating changes: leader-folower model ..................................................................................................... 74
Replicating changes: leaderless model ............................................................................................................ 75
Quorums for reading and writing..................................................................................................................... 77
Quorum consistency ......................................................................................................................................... 78
Sloppy Quorums ............................................................................................................................................... 78
Distributed data: partitioning/sharding ........................................................................................................... 79
Waarom partitioneren? ................................................................................................................................... 79
4
Inhoudsopgave
Hoofdstuk 0: Introduction .............................................................................................................................. 6
Systeemcomponenten ........................................................................................................................................ 7
Systeemontwerp in bedrijven en deployen van een systeem ............................................................................. 7
Systeemarchitectuur .......................................................................................................................................... 8
Systeemrequirements......................................................................................................................................... 8
4 + 1 view model van softwarearchitectuur ....................................................................................................... 8
Reactive Manifesto .......................................................................................................................................... 11
Hoofdstuk 1: Microservices .......................................................................................................................... 13
Architectural style ............................................................................................................................................ 13
Wat is architecturale stijl ................................................................................................................................. 13
4 Architecturale stijlen ..................................................................................................................................... 13
Microservices ................................................................................................................................................... 16
Wat is een service ............................................................................................................................................ 16
Wat is een microservice ................................................................................................................................... 16
What is loosely coupled? .................................................................................................................................. 17
Nadelen/Challenges van het MicroService pattern .......................................................................................... 17
Decompositie.................................................................................................................................................... 18
Stap 1: identificeren en begrijpen van systeemoperaties................................................................................. 18
Stap 2: Identificeren van services door business capabilities ........................................................................... 19
Stap 3: Definiëren van service API’s en collaborations ..................................................................................... 20
Finaliseren van de services ............................................................................................................................... 21
Problemen bij decompositie ............................................................................................................................. 21
Interactiestijlen tussen micro-services ............................................................................................................. 22
Interactie is een logisch en technisch probleem ............................................................................................... 22
Interactiestijlen ................................................................................................................................................ 22
Synchrone communicatie: gRPC ....................................................................................................................... 24
Handling partial failure .................................................................................................................................... 24
Bulkhead: limiet zetten op het aantal uitgaande requests .............................................................................. 25
Circuit breaker .................................................................................................................................................. 25
Fallback strategy to recover from failing call ................................................................................................... 25
Asynchrone communicatie: Messaging ............................................................................................................ 25
Messaging supports all interaction styles ........................................................................................................ 26
Specification for messaging-based service API ................................................................................................ 26
1
, Message broker technologieën ........................................................................................................................ 27
Design issues: competing consumers ............................................................................................................... 27
Hoe lost Kafka die problemen op? ................................................................................................................... 27
Kafka at work: voorbeeld MMOG .................................................................................................................... 28
Design issues: handling duplicate messages .................................................................................................... 29
Synchrone interactie reduceert beschikbaarheid vd applicatie ........................................................................ 30
Interactie is orthogonaal (niet gelijk aan) de gebruike communicatie............................................................. 30
Eliminating Synchronous interaction................................................................................................................ 31
Saga’s ............................................................................................................................................................... 32
Transactiegaranties in een monoliet (monolithic system) ............................................................................... 32
Distributed transactions: 2PC (2 phase commit) (bij microservices) ................................................................ 32
Saga: sequentie van lokale transacties ............................................................................................................ 33
Transactionele messaging ................................................................................................................................ 33
Compensating transactions ............................................................................................................................. 33
Voorbeeld: rejecting an order .......................................................................................................................... 34
Concurrent Saga’s ............................................................................................................................................ 34
Saga coordination: hoe wordt de saga uitgevoerd? ........................................................................................ 34
State Machine model ....................................................................................................................................... 35
Business logica van een enkele microservice ................................................................................................... 36
Business logic organization .............................................................................................................................. 36
Building blocks for a good domain model ........................................................................................................ 37
= Een klasse: heeft ............................................................................................................................................ 37
Putting everything in place .............................................................................................................................. 39
Example: restaurant order management ......................................................................................................... 39
Queries in a microservice architecture ............................................................................................................. 40
The API composition pattern ............................................................................................................................ 40
Design issue: wie speelt de rol van API composer? .......................................................................................... 40
Moeilijkheden bij queries naar 1 service .......................................................................................................... 41
Command-Query responsibility segregation: oplossing voor moeilijkheden ................................................... 41
Hoofdstuk 2: Deployment and orchestratie .................................................................................................. 43
From the bottom to the top: Development naar productie ............................................................................. 43
Making your service production ready ............................................................................................................. 44
Configureerbare services .................................................................................................................................. 44
Push-based externalized configuration ............................................................................................................ 44
Pull-based externalized configuration .............................................................................................................. 44
Observable services .......................................................................................................................................... 44
Log aggregation ............................................................................................................................................... 45
Distributed Tracing ........................................................................................................................................... 45
2
,Audit logging .................................................................................................................................................... 45
Application metrics .......................................................................................................................................... 46
Containers als een unit voor deployment ......................................................................................................... 46
Probleemstatement: development to operations ............................................................................................ 46
Virtual machine images als deployable artifacts ............................................................................................. 46
Host virtualisatie: VM’s en containers ............................................................................................................. 47
Linux containers technology stack ................................................................................................................... 47
Functies van Linux Kernel: Linux Control Groups (CGROUPS) ........................................................................... 47
Functie van de Linux Kernel: Linux namespaces ............................................................................................... 48
Functie van Linux Kernel: changing root .......................................................................................................... 48
Functie van Linux Kernel: Secure containers .................................................................................................... 48
Wat is Docker? ................................................................................................................................................. 48
Docker vs Linux containers ............................................................................................................................... 49
Docker architectuur .......................................................................................................................................... 49
Building images from Dockerfiles ..................................................................................................................... 49
Images layers ................................................................................................................................................... 50
Container vs image........................................................................................................................................... 50
Building an image ............................................................................................................................................ 50
Union file system .............................................................................................................................................. 50
Container Orchestration................................................................................................................................... 51
Wat doet een Orchestration framework? ........................................................................................................ 51
Kubernetes architectuur ................................................................................................................................... 51
Kubernetes concepten: PODS en deployment objecten ................................................................................... 52
Multicontainer pods ......................................................................................................................................... 52
Service discovery: the need of a service registry .............................................................................................. 52
Hoe gaan clients de services in het registry vinden? ........................................................................................ 53
Servcie Discovery in Kubernetes? ..................................................................................................................... 53
Load balancer ................................................................................................................................................... 53
Rolling updates: zero-downtime ...................................................................................................................... 54
Andere deployment operaties met een load balancer ..................................................................................... 54
Dimensions of cloud computing ....................................................................................................................... 55
Essentiële karakterisitieken .............................................................................................................................. 55
Stakeholders (service models) .......................................................................................................................... 55
De rol van de cloud provider ............................................................................................................................ 55
Multi-tenant homogeneous pooled resources ................................................................................................. 56
Cloud service models: X as a service (fill in the X) ............................................................................................ 56
Deployment models (nieuwe pijl, de pijl omhoog) ........................................................................................... 58
Elastic Auto-scaling .......................................................................................................................................... 59
3
, Cloud user objectives........................................................................................................................................ 59
Statisch vs elastisch schalen ............................................................................................................................. 59
Verschillende soorten workload ....................................................................................................................... 60
Types of scaling ................................................................................................................................................ 60
Verticaal schalen .............................................................................................................................................. 61
Horizontaal schalen.......................................................................................................................................... 61
Autoscaling....................................................................................................................................................... 61
Case study; auto-scaling in Amazon EC2 .......................................................................................................... 62
Standy pooling ................................................................................................................................................. 62
Scaling guidelines ............................................................................................................................................. 63
Multisite deployment ....................................................................................................................................... 63
Two stage load balancing ................................................................................................................................ 63
Example: Google Compute Engine ................................................................................................................... 64
Hoofdstuk 3: Data intensieve systemen: Deel 1 ............................................................................................... 66
Challenges of modern data .............................................................................................................................. 66
Data-intensieve systemen ................................................................................................................................ 66
Datamodellen................................................................................................................................................... 66
NoSQL ............................................................................................................................................................... 68
Data storage and retrieval ............................................................................................................................... 69
Twee soorten storage engines ......................................................................................................................... 69
Hoe storage engines beoordelen? .................................................................................................................... 69
B-tree index for page-based storage ................................................................................................................ 70
Log-structured storage engines ....................................................................................................................... 70
Betrouwbaarheid van een DB .......................................................................................................................... 71
Comparing B-trees to LSM-trees ...................................................................................................................... 71
Ster schema voor analyse................................................................................................................................. 72
Relationele data opgeslagen per kolom, niet per rij ........................................................................................ 72
Distributed data ............................................................................................................................................... 73
Waarom data distribueren? ............................................................................................................................. 73
Patterns voor datadistributie ........................................................................................................................... 73
Distribute data: replicatie ................................................................................................................................ 74
Replicating changes: leader-folower model ..................................................................................................... 74
Replicating changes: leaderless model ............................................................................................................ 75
Quorums for reading and writing..................................................................................................................... 77
Quorum consistency ......................................................................................................................................... 78
Sloppy Quorums ............................................................................................................................................... 78
Distributed data: partitioning/sharding ........................................................................................................... 79
Waarom partitioneren? ................................................................................................................................... 79
4
