CS 643 CLOUD COMPUTING FINAL STUDYGUIDE QUESTIONS AND CORRECT
ANSWERS
Question:
1. What is the advantage of using the record append function instead of the write function in
GFS? Are GFS replicas guaranteed to have the same content if only record append functions
are used to write data? Justify your answers.
Answer:
Since the files are huge, it's quite often the case that many clients need to write them
concurrently. These writes can lead to inconsistent data (or undefined). Locking slows everything
down when the clients write at different offsets. The situation is even worse if distributed locking
is required (when using multiple replicas). Record appends provide a solution to this problem, but
it forces the clients to use only appends. These appends are guaranteed atomicity and implicitly
lead to defined file regions across replicas: it's impossible to overwrite each other's data. A
primary replica decides a sequential order of applying the updates at all replicas. Replicas can be
different because some appends can fail at some replica - therefore the entire append fails. The
client re-tries and eventually succeeds to write the record at the same offset in all replicas.
At-least-once semantics can lead to duplicates/fragments/padding in some replicas.
Question:
2. Give an example of a situation that causes a Bigtable tablet server to shutdown. Why
must the server shut down in this situation?
Answer:
Each tablet server must create a file and hold a lock on it (from Chubby) to show the master that
it's still alive. Each lock is allocated for a short period of time (i.e., a lease). If the lease expires,
the server has to renew it. Let's assume the network link of the tablet server goes down for a
while. During this time, the master tries to contact the server and cannot do it. Therefore, it
assumes that the tablet server crashed and decides to re-allocate the tablets. If the lock lease
expired, the master can acquire it and remove the file associated with the tablet server. When
the tablet server's link comes back up, it sees that its file was deleted and shuts itself down. It
does this because only one tablet server can be responsible doesn't do it, it can lead to
inconsistent data (due to stale data, concurrent writes, etc.).
Question:
3. Describe two problems associated with data partitioning in distributed systems. Also,
describe the solution provided by Dynamo for both problems.
Answer:
Load balancing/scalability. The idea is to distribute the data evenly across the set of nodes.
Lookup time. The idea is to be able to find the data as fast as possible with minimum amount of
meta-data stored at nodes. The solution provided by Dynamo is consistent hashing + virtual
nodes + routing table entries to all other nodes. Consistent hashing represents data as (key,
value)pairs, and then evenly distributes the keys in the system. The nodes can find the data in
O(1) because they maintain routing for all other nodes. However, the load balancing might not be
very good with a low number of nodes in the system. Additionally, load balancing may suffer from
hardware heterogeneity. An optimization, used in Dynamo, is to create virtual nodes and assign a
certain number of virtual nodes to each physical node.
,Question:
4. How does virtualization improve system security? How can it improve load
balancing/fault-tolerance? What are the two main conditions that must be satisfied to
provide load balancing/fault-tolerance for Internet servers?
Answer:
Virtualization improves system security because each application can be run into its own virtual
machine (i.e., guest OS). If the application or its OS crashes (or they are compromised), they
won't affect other applications/OSs. Load balancing/fault-tolerance can be improved through live
migration. It's easier to transfer entire VMs than individual processes. Ensure that the server
maintains the same IP address and have a minimal downtime when moving the VM such that TCP
connections don't time out.
Question:
5. Explain the similarities and differences between the mechanisms used for memory
virtualization in VMware and Xen.
Answer:
Similarity: both add one more level of indirection, i.e., physical pages (what the OS believes to be
physical memory) to machine pages (real physical memory as seen by VMM). Difference: In
VMware, VMM maintains shadow page tables for all guest OSs. In regular OSs, TLB points to the
OS page table. In VMware, it points to the shadow page table which contains composite
mappings. These shadow page tables are updated every time the OS page tables are updated.
Practically, the TLB contains mapping between virtual pages of the running guest OS and
machine pages. Every TLB miss goes to the shadow page tables to find the mappings. This has a
high overhead because it may require a lookup in the OS page table as well. In Xen, each guest
OS allocates and manages its own memory (allocated at create time by VMM and potentially
updated using the balloon driver). VMM points the TLB to the running guest OS page table. Any
read from the page table is fast because it uses the OS page table. This achieves better
performance than shadow page tables. However, page table updates trap into VMM for validation
(e.g., OS maps page that it owns). This (trap for updates but not for reads) can be done because
guest OSs are modified
Question:
6. How can an attacker determine the load of a web server running in the AWS cloud if this
attacker owns a VM on the same physical machine with the webserver? tage of storage area
networks (SANs) for large-scale data storage, and how does it differ from distributed file
systems? Provide an example of a storage area network solution.
Answer:
The attacker can use externally-generated load to determine the correlation between the load
and its own cache-based load measurement. The cache-based load measurements estimate the
effect of another VM load on the read performance of the current VM. Essentially, higher load on
the co-located VM leads to more cache lines of the attacker being evicted, thus its read
performance decreases. data storage as they allow remote storage devices to be attached to
servers through fast network communication, such as fiber optics. SANs provide block-level
abstraction and good reliability, often using techniques like RAID. An example of a storage area
network solution is HP's StorageWorks, which supports up to 247PB of storage, although it comes
, with significant cost implications.
Question:
7. Describe the main features of remote service and caching in the context of distributed file
systems. How does caching contribute to reducing server load and improving scalability?
Answer:
Remote service involves implementing all file operations on the server, communicating through
Remote Procedure Calls (RPC). This is particularly applicable for scenarios with a large amount of
write activity. Caching, on the other hand, involves caching files locally, reducing the need for
frequent server contact, thus lowering server load and network traffic. The challenge in caching
lies in determining how much of a file to cache and ensuring cache coherence.
Question:
8. Differentiate between stateless and stateful services in distributed file systems. Highlight
the advantages and disadvantages of each approach, and provide examples of situations
where each might be more suitable.
Answer:
Stateless services avoid storing state information on the server by making each request
self-contained. This approach simplifies recovery from failures but offers poor support for locking
or synchronization. In contrast, stateful services involve the server fetching information about a
file from disk upon an open request, storing it in server memory, and returning a unique
connection identifier for subsequent accesses. While stateful services offer increased
performance, recovery from failures can be more expensive.
Question:
9. Explain the concept of replication in distributed file systems. How does replication
contribute to availability and cuss the challenges associated with maintaining consistency
among replicas.
Answer:
Replication involves creating replicas of the same file on failure-independent machines. This
improves availability and can shorten service time. Replicas must be distinguished from each
must be reflected on all other replicas to maintain consistency.
Question:
10. Provide an overview of Sun's Network File System (NFS). How does NFS implement
remote directories, and what is the significance of mounting a remote directory onto a local
directory?
Answer:
Sun's Network File System (NFS) allows a remote directory to be mounted onto a local directory.
This enables the remote directory to appear as part of the local file system. NFS provides a
transparent way to access files on a remote server, and the mounted directory may contain other
mounted directories.
Question:
11. Discuss the implementation details of Sun's NFS, focusing on network communication,
the stateless service approach, and any caching mechanisms involved. How does NFS
ANSWERS
Question:
1. What is the advantage of using the record append function instead of the write function in
GFS? Are GFS replicas guaranteed to have the same content if only record append functions
are used to write data? Justify your answers.
Answer:
Since the files are huge, it's quite often the case that many clients need to write them
concurrently. These writes can lead to inconsistent data (or undefined). Locking slows everything
down when the clients write at different offsets. The situation is even worse if distributed locking
is required (when using multiple replicas). Record appends provide a solution to this problem, but
it forces the clients to use only appends. These appends are guaranteed atomicity and implicitly
lead to defined file regions across replicas: it's impossible to overwrite each other's data. A
primary replica decides a sequential order of applying the updates at all replicas. Replicas can be
different because some appends can fail at some replica - therefore the entire append fails. The
client re-tries and eventually succeeds to write the record at the same offset in all replicas.
At-least-once semantics can lead to duplicates/fragments/padding in some replicas.
Question:
2. Give an example of a situation that causes a Bigtable tablet server to shutdown. Why
must the server shut down in this situation?
Answer:
Each tablet server must create a file and hold a lock on it (from Chubby) to show the master that
it's still alive. Each lock is allocated for a short period of time (i.e., a lease). If the lease expires,
the server has to renew it. Let's assume the network link of the tablet server goes down for a
while. During this time, the master tries to contact the server and cannot do it. Therefore, it
assumes that the tablet server crashed and decides to re-allocate the tablets. If the lock lease
expired, the master can acquire it and remove the file associated with the tablet server. When
the tablet server's link comes back up, it sees that its file was deleted and shuts itself down. It
does this because only one tablet server can be responsible doesn't do it, it can lead to
inconsistent data (due to stale data, concurrent writes, etc.).
Question:
3. Describe two problems associated with data partitioning in distributed systems. Also,
describe the solution provided by Dynamo for both problems.
Answer:
Load balancing/scalability. The idea is to distribute the data evenly across the set of nodes.
Lookup time. The idea is to be able to find the data as fast as possible with minimum amount of
meta-data stored at nodes. The solution provided by Dynamo is consistent hashing + virtual
nodes + routing table entries to all other nodes. Consistent hashing represents data as (key,
value)pairs, and then evenly distributes the keys in the system. The nodes can find the data in
O(1) because they maintain routing for all other nodes. However, the load balancing might not be
very good with a low number of nodes in the system. Additionally, load balancing may suffer from
hardware heterogeneity. An optimization, used in Dynamo, is to create virtual nodes and assign a
certain number of virtual nodes to each physical node.
,Question:
4. How does virtualization improve system security? How can it improve load
balancing/fault-tolerance? What are the two main conditions that must be satisfied to
provide load balancing/fault-tolerance for Internet servers?
Answer:
Virtualization improves system security because each application can be run into its own virtual
machine (i.e., guest OS). If the application or its OS crashes (or they are compromised), they
won't affect other applications/OSs. Load balancing/fault-tolerance can be improved through live
migration. It's easier to transfer entire VMs than individual processes. Ensure that the server
maintains the same IP address and have a minimal downtime when moving the VM such that TCP
connections don't time out.
Question:
5. Explain the similarities and differences between the mechanisms used for memory
virtualization in VMware and Xen.
Answer:
Similarity: both add one more level of indirection, i.e., physical pages (what the OS believes to be
physical memory) to machine pages (real physical memory as seen by VMM). Difference: In
VMware, VMM maintains shadow page tables for all guest OSs. In regular OSs, TLB points to the
OS page table. In VMware, it points to the shadow page table which contains composite
mappings. These shadow page tables are updated every time the OS page tables are updated.
Practically, the TLB contains mapping between virtual pages of the running guest OS and
machine pages. Every TLB miss goes to the shadow page tables to find the mappings. This has a
high overhead because it may require a lookup in the OS page table as well. In Xen, each guest
OS allocates and manages its own memory (allocated at create time by VMM and potentially
updated using the balloon driver). VMM points the TLB to the running guest OS page table. Any
read from the page table is fast because it uses the OS page table. This achieves better
performance than shadow page tables. However, page table updates trap into VMM for validation
(e.g., OS maps page that it owns). This (trap for updates but not for reads) can be done because
guest OSs are modified
Question:
6. How can an attacker determine the load of a web server running in the AWS cloud if this
attacker owns a VM on the same physical machine with the webserver? tage of storage area
networks (SANs) for large-scale data storage, and how does it differ from distributed file
systems? Provide an example of a storage area network solution.
Answer:
The attacker can use externally-generated load to determine the correlation between the load
and its own cache-based load measurement. The cache-based load measurements estimate the
effect of another VM load on the read performance of the current VM. Essentially, higher load on
the co-located VM leads to more cache lines of the attacker being evicted, thus its read
performance decreases. data storage as they allow remote storage devices to be attached to
servers through fast network communication, such as fiber optics. SANs provide block-level
abstraction and good reliability, often using techniques like RAID. An example of a storage area
network solution is HP's StorageWorks, which supports up to 247PB of storage, although it comes
, with significant cost implications.
Question:
7. Describe the main features of remote service and caching in the context of distributed file
systems. How does caching contribute to reducing server load and improving scalability?
Answer:
Remote service involves implementing all file operations on the server, communicating through
Remote Procedure Calls (RPC). This is particularly applicable for scenarios with a large amount of
write activity. Caching, on the other hand, involves caching files locally, reducing the need for
frequent server contact, thus lowering server load and network traffic. The challenge in caching
lies in determining how much of a file to cache and ensuring cache coherence.
Question:
8. Differentiate between stateless and stateful services in distributed file systems. Highlight
the advantages and disadvantages of each approach, and provide examples of situations
where each might be more suitable.
Answer:
Stateless services avoid storing state information on the server by making each request
self-contained. This approach simplifies recovery from failures but offers poor support for locking
or synchronization. In contrast, stateful services involve the server fetching information about a
file from disk upon an open request, storing it in server memory, and returning a unique
connection identifier for subsequent accesses. While stateful services offer increased
performance, recovery from failures can be more expensive.
Question:
9. Explain the concept of replication in distributed file systems. How does replication
contribute to availability and cuss the challenges associated with maintaining consistency
among replicas.
Answer:
Replication involves creating replicas of the same file on failure-independent machines. This
improves availability and can shorten service time. Replicas must be distinguished from each
must be reflected on all other replicas to maintain consistency.
Question:
10. Provide an overview of Sun's Network File System (NFS). How does NFS implement
remote directories, and what is the significance of mounting a remote directory onto a local
directory?
Answer:
Sun's Network File System (NFS) allows a remote directory to be mounted onto a local directory.
This enables the remote directory to appear as part of the local file system. NFS provides a
transparent way to access files on a remote server, and the mounted directory may contain other
mounted directories.
Question:
11. Discuss the implementation details of Sun's NFS, focusing on network communication,
the stateless service approach, and any caching mechanisms involved. How does NFS
