Submission Deadline: 3^rd November 2025, 18:00 (submit through Canvas)

Objective

The purpose of this lab is to evaluate your basic understanding of cloud service models and their implementations. For the purpose of this lab, you can use any cloud provider.

Answer the questions (include necessary screenshots of code and result) in a PDF Document (YourStudentID.pdf, e.g., A0123456A.pdf) and submit to Canvas. A penalty of 1 mark would be applied to any submission not following the naming convention. Do not worry about the auto numbering Canvas adds to any resubmissions.

Part 1: Service Models (4 marks)

a) Give one advantage and one disadvantage of using the MapReduce programming model over not using it.

b) Your workplace wishes to make the work environment a bit more fun. They are going to provide advanced collaboration services, such as video, chat, and an online whiteboard. Given the cost of hosting the infrastructure for this, they have decided that a cloud-based approach is the way to go. Which cloud computing deployment model would best suit your company’s needs? Explain your answer.

Part 2: MapReduce It! (6 marks)

To display the final results, you will use a full-stack web application with a front-end and a back-end, the latter including the MapReduce component. In this part, you will develop a web application that internally uses MapReduce to compute the total number of followers and followees for each user in a Twitter dataset. The dataset is provided to you in the file twitter combined.txt (taken from https://snap.stanford.edu/data/ egonets-Twitter.html). The format of the file is as follows.

Each line of the file contains two user IDs.

Line 1 of the file contains 214328887 34428380. This implies that user 214328887 follows user 34428380.

1. Front-End (2 marks)

The front-end fetches data from the back-end by calling backend APIs and presents user statistics in the following format:

UserID1 has Numberof followers

UserID1 has Numberof followees

UserID2 has Numberof followers

UserID2 has Numberof followees

UserID3 has Numberof followers

UserID3 has Numberof followees

UserIDN has Numberof followers

UserIDN has Numberof followees where a “followee” of UserIDN represents a user that is followed by UserIDN.

In your submission, you should show the results for user 214328887 and user 107830991.

You need to provide a short description (max 50 words) of the set-up of your front-end. For example, about the technologies used and how you connect to the backend.

2. Back-End (2 marks)

The back-end acts as the core hub, managing the provided dataset (e.g., from AWS S3, Google Cloud Storage, Azure Blob Storage, etc.) and coordinating with the frontend and MapReduce components (e.g., AWS EMR, Google Dataproc, Azure HDInsight, etc.). It retrieves MapReduce results (follower and followee counts) and serves them to the front-end via APIs.

You need to provide a short description (max 100 words) of the set-up of your backend. For example, about the technologies used and how you use the MapReduce component.

3. MapReduce Component (2 marks)

This component uses cloud-based MapReduce services (e.g., AWS EMR, Google Dataproc, Azure HDInsight, etc.), which support Hadoop and frameworks like Spark, for processing the Twitter dataset in parallel, calculating follower and followee counts. This will contain the Map() and Reduce() functions that perform the bulk of the necessary tasks.

You configure it by selecting a cloud service, specifying the dataset location in cloud storage (e.g., AWS S3, Google Cloud Storage, Azure Blob Storage, etc.), and uploading the Map() and Reduce() scripts (e.g., in Python) to process the input line-by-line using Hadoop or similar frameworks (e.g., Spark, etc.). The cloud provider manages the distribution across nodes, showcasing scalable computing.

In your submission, please include the source code for the Map() and Reduce() functions as well as a few screenshots to show that you run your solution on the cloud.

data :-
https://prime-management-system-bucket.s3.amazonaws.com/twittercombined-fa0cdaca-1ce9-4c92-859e-4798233fcf58.txt