Advanced Data Systems for Smart Services Summary
A summary of 45+ articles and book chapters in advanced data systems
Author: Martijn C. Paulussen
University: Maastricht University School of Business and Economics
Master: MSc Business Intelligence & Smart Services
Course: [EBC4224] Advanced Data Systems for Smart Services
School of Business and Economics
MSc Business Intelligence & Smart Services
© 2018 Martijn Paulussen - Maastricht University School of Business and Economics
Nothing in this publication may be reproduced and/or made public by means of printing, offset, photocopy or
microfilm or in any digital, electronic, optical or any other form without the prior written permission of the
owner of the copyright.
,Table of Contents
1. Introduction to Advanced Data Systems for Smart Services (lecture) ...................................................2
1.1. Presentation: Advanced Data Systems for Smart Services & Conceptualizing smart service
systems & [Book Chapter 1] Designing Data Intensive Applications by Martin Klepmann .....................2
2. Relational Data Model and SQL Query Language .................................................................................4
2.1. Presentation: Databases ..................................................................................................................4
2.2. Database Fundamentals ..................................................................................................................7
3. Map-Reduce and Hadoop .......................................................................................................................8
3.1. MapReduce Simplified Data Processing on Large Clusters ...........................................................8
3.2. Parallel Data Processing with MapReduce A Survey.....................................................................9
3.3. The Hadoop Distributed File System ...........................................................................................10
3.4. Presentation: Examples and Exercise ...........................................................................................11
4. Some notions of distributed systems ....................................................................................................13
4.1. Eventually Consistent ...................................................................................................................13
4.2. Consistency in DS ........................................................................................................................14
4.3. Time and State in DS ....................................................................................................................15
4.4. Consistent Hashing (Rings) ..........................................................................................................17
4.5. Presentation: Replication in Distributed Systems.........................................................................18
5. No-SQL databases ................................................................................................................................22
5.1. NoSQL Systems for Big Data Management .................................................................................22
5.2. Scalable SQL and NoSQL Data Stores ........................................................................................24
5.3. Security Issues in NoSQL Databases ...........................................................................................25
5.4. Dynamo Amazon's Highly Available Key-value Store ................................................................25
6. Semantic Web.......................................................................................................................................27
6.1. The Semantic Web .......................................................................................................................27
6.2. A New Look at the Semantic Web ...............................................................................................28
6.3. The FAIR Guiding Principles for scientific data management and stewardship ..........................28
6.4. Semantic Web in Data Mining and knowledge discovery A compressive survey .......................29
7. Blockchain ............................................................................................................................................31
7.1. Bitcoin A Peer-to-Peer Electronic Cash System ..........................................................................31
7.2. Blockchains and Smart Contracts for the Internet of Things .......................................................32
7.3. A Case Study for Blockchain in Healthcare - MedRec ...............................................................33
8. m-Health and Mobile Cloud Computing ..............................................................................................34
8.1. A survey of mobile cloud computing architecture, applications, and approaches .......................34
8.2. Mobile Health Technology Evaluation.........................................................................................37
8.3. COMMODITY A Smart e-Health Environment for Diabetes Management ................................37
8.4. Presentation: Autonomous systems and Multi-Agent Systems (MAS) ........................................38
1
, 1. Introduction to Advanced Data Systems for Smart Services (lecture)
Papers discussed are:
- Presentation: Advanced Data Systems For Smart Services
- Conceptualizing smart service systems
- Book Chapter: Data-Intensive Applications
1.1. Presentation: Advanced Data Systems for Smart Services &
Conceptualizing smart service systems &
[Book Chapter 1] Designing Data Intensive Applications by Martin Klepmann
Advanced Data Systems are:
- Data intensive
o The amount of data
o The complexity of data
o The speed at which it is changing
o Databases: Store data for later use
o Caches: Remember results for faster reads
o Search indexes: search by keywords
o Stream processing: Handle processes asynchronously
o Batch processing: Large amounts of accumulated data
- Distributed solutions
- Multiple actors
Advanced Data Systems should be:
- Reliable: The system works correctly even in the face of adversity, tolerates user mistakes,
performance and prevents abuse.
o Fault: A component deviating from its specifications, caused by poor error handling. You
can prevent faults by including tests with faults to ensure correct handling when it
happens. (e.g. Hardware, Software, Human)
o Failure: The system as a whole stops providing the required service to the user
o Systems that anticipate faults are called fault-tolerant or resilient.
o Why? Safety, Loss of productivity, Damage to reputation
- Scalable: As the system grows (in data, traffic or complexity), there is a way of dealing with that
growth or load increase.
o Keep data in cache for faster and cheaper reads and requests of for example timelines
(twitter).
o Percentiles are best to understand the average performance (median = 50%)
o Scaling up: Vertical scaling more powerful machines
o Scaling out: Horizontal scaling more and multiple smaller machines
o Elastic systems: can automatically increase computing resources.
- Maintainable: Maintaining current behavior and adapting the system to new use cases.
o The cost of software is not in its initial development, but in its ongoing maintenance.
o Design Principles:
Operability: Easy for teams to keep the system running smoothly.
Simplicity: Easy for new engineers to understand the system/remove complexity.
Extensibility: Make it easy for engineers to make changes to the system in the
future, adapting it for unanticipated use cases.
2
, Advanced Data Systems questions can be:
- How to ensure that the data remains correct and complete, even when things go wrong?
- How do you provide consistently good performance to clients?
- How do you scale to handle an increase in load?
- What does a good API for the service look like?
Smart Systems:
- Smart service systems bring a customer’s and provider’s perspective on value creation with a
smart product. Which can consist of connected sensors or Internet of Things devices.
- Key features:
o Unique identifier: addressable by other products.
o Localized: “know” their location and can be localized and traced.
o Connected: share data and functionality with other devices.
o Sensors: obtain physical data in their proximity.
o Storage: store data locally which can be status, usage or contextual.
o Computation: Compute and give results for autonomous behavior.
o Actuators: share data and functionality with other IoT devices.
o Interfaces: Intractable with humans (not only desktop-wise).
- Architecture and smart services:
o Core properties: Awareness and connectivity.
o Basic properties: Single shared identity and different interpretability.
o Offer contextual and preemptive services based on intelligence.
o Create value in use energy consumption optimizer
3
A summary of 45+ articles and book chapters in advanced data systems
Author: Martijn C. Paulussen
University: Maastricht University School of Business and Economics
Master: MSc Business Intelligence & Smart Services
Course: [EBC4224] Advanced Data Systems for Smart Services
School of Business and Economics
MSc Business Intelligence & Smart Services
© 2018 Martijn Paulussen - Maastricht University School of Business and Economics
Nothing in this publication may be reproduced and/or made public by means of printing, offset, photocopy or
microfilm or in any digital, electronic, optical or any other form without the prior written permission of the
owner of the copyright.
,Table of Contents
1. Introduction to Advanced Data Systems for Smart Services (lecture) ...................................................2
1.1. Presentation: Advanced Data Systems for Smart Services & Conceptualizing smart service
systems & [Book Chapter 1] Designing Data Intensive Applications by Martin Klepmann .....................2
2. Relational Data Model and SQL Query Language .................................................................................4
2.1. Presentation: Databases ..................................................................................................................4
2.2. Database Fundamentals ..................................................................................................................7
3. Map-Reduce and Hadoop .......................................................................................................................8
3.1. MapReduce Simplified Data Processing on Large Clusters ...........................................................8
3.2. Parallel Data Processing with MapReduce A Survey.....................................................................9
3.3. The Hadoop Distributed File System ...........................................................................................10
3.4. Presentation: Examples and Exercise ...........................................................................................11
4. Some notions of distributed systems ....................................................................................................13
4.1. Eventually Consistent ...................................................................................................................13
4.2. Consistency in DS ........................................................................................................................14
4.3. Time and State in DS ....................................................................................................................15
4.4. Consistent Hashing (Rings) ..........................................................................................................17
4.5. Presentation: Replication in Distributed Systems.........................................................................18
5. No-SQL databases ................................................................................................................................22
5.1. NoSQL Systems for Big Data Management .................................................................................22
5.2. Scalable SQL and NoSQL Data Stores ........................................................................................24
5.3. Security Issues in NoSQL Databases ...........................................................................................25
5.4. Dynamo Amazon's Highly Available Key-value Store ................................................................25
6. Semantic Web.......................................................................................................................................27
6.1. The Semantic Web .......................................................................................................................27
6.2. A New Look at the Semantic Web ...............................................................................................28
6.3. The FAIR Guiding Principles for scientific data management and stewardship ..........................28
6.4. Semantic Web in Data Mining and knowledge discovery A compressive survey .......................29
7. Blockchain ............................................................................................................................................31
7.1. Bitcoin A Peer-to-Peer Electronic Cash System ..........................................................................31
7.2. Blockchains and Smart Contracts for the Internet of Things .......................................................32
7.3. A Case Study for Blockchain in Healthcare - MedRec ...............................................................33
8. m-Health and Mobile Cloud Computing ..............................................................................................34
8.1. A survey of mobile cloud computing architecture, applications, and approaches .......................34
8.2. Mobile Health Technology Evaluation.........................................................................................37
8.3. COMMODITY A Smart e-Health Environment for Diabetes Management ................................37
8.4. Presentation: Autonomous systems and Multi-Agent Systems (MAS) ........................................38
1
, 1. Introduction to Advanced Data Systems for Smart Services (lecture)
Papers discussed are:
- Presentation: Advanced Data Systems For Smart Services
- Conceptualizing smart service systems
- Book Chapter: Data-Intensive Applications
1.1. Presentation: Advanced Data Systems for Smart Services &
Conceptualizing smart service systems &
[Book Chapter 1] Designing Data Intensive Applications by Martin Klepmann
Advanced Data Systems are:
- Data intensive
o The amount of data
o The complexity of data
o The speed at which it is changing
o Databases: Store data for later use
o Caches: Remember results for faster reads
o Search indexes: search by keywords
o Stream processing: Handle processes asynchronously
o Batch processing: Large amounts of accumulated data
- Distributed solutions
- Multiple actors
Advanced Data Systems should be:
- Reliable: The system works correctly even in the face of adversity, tolerates user mistakes,
performance and prevents abuse.
o Fault: A component deviating from its specifications, caused by poor error handling. You
can prevent faults by including tests with faults to ensure correct handling when it
happens. (e.g. Hardware, Software, Human)
o Failure: The system as a whole stops providing the required service to the user
o Systems that anticipate faults are called fault-tolerant or resilient.
o Why? Safety, Loss of productivity, Damage to reputation
- Scalable: As the system grows (in data, traffic or complexity), there is a way of dealing with that
growth or load increase.
o Keep data in cache for faster and cheaper reads and requests of for example timelines
(twitter).
o Percentiles are best to understand the average performance (median = 50%)
o Scaling up: Vertical scaling more powerful machines
o Scaling out: Horizontal scaling more and multiple smaller machines
o Elastic systems: can automatically increase computing resources.
- Maintainable: Maintaining current behavior and adapting the system to new use cases.
o The cost of software is not in its initial development, but in its ongoing maintenance.
o Design Principles:
Operability: Easy for teams to keep the system running smoothly.
Simplicity: Easy for new engineers to understand the system/remove complexity.
Extensibility: Make it easy for engineers to make changes to the system in the
future, adapting it for unanticipated use cases.
2
, Advanced Data Systems questions can be:
- How to ensure that the data remains correct and complete, even when things go wrong?
- How do you provide consistently good performance to clients?
- How do you scale to handle an increase in load?
- What does a good API for the service look like?
Smart Systems:
- Smart service systems bring a customer’s and provider’s perspective on value creation with a
smart product. Which can consist of connected sensors or Internet of Things devices.
- Key features:
o Unique identifier: addressable by other products.
o Localized: “know” their location and can be localized and traced.
o Connected: share data and functionality with other devices.
o Sensors: obtain physical data in their proximity.
o Storage: store data locally which can be status, usage or contextual.
o Computation: Compute and give results for autonomous behavior.
o Actuators: share data and functionality with other IoT devices.
o Interfaces: Intractable with humans (not only desktop-wise).
- Architecture and smart services:
o Core properties: Awareness and connectivity.
o Basic properties: Single shared identity and different interpretability.
o Offer contextual and preemptive services based on intelligence.
o Create value in use energy consumption optimizer
3
