Question 1 

Objectives:
● Understand datasets with a data scientist mindset.
● Understand and design computation logic and routines in Python.
● Assess the use of Python only and Python data structures to perform extract, load, and transformation operations.
● Assess the use of Pandas data frame to perform extract, load, transformation, and calculation
operations.
● Structure code in appropriate methods (functions), looping, and conditions.
● Conduct visualization in an appropriate way.

The dataset in question provides a rich overview of Housing and Development Board (HDB) flat transactions in Singapore. Derived from the national database managed by Singapore’s open data initiative.

The data captured includes vital information such as the resale price, flat type, address, lease commencement date, and floor area, among other details. These elements allow for robust analysis on a multitude of aspects such as price trends and geographical price disparities. You may refer to more information at `https://data.gov.sg/dataset/resale-flat-prices`.

Additionally, this dataset provides an invaluable resource for understanding the evolution of Singapore’s public housing landscape, the preferences of the populace, and market dynamics over time. As such, it is an essential tool for policymakers, real estate professionals, urban planners, and researchers studying Singapore’s unique public housing model.

By addressing the given tasks, you will gain data analysis competencies, including data reprocessing and manipulation, fundamental for preparing and managing datasets. Additionally,

you’ll enhance your ability to comprehend data relationships through the practice of creating data visualizations and executing correlation analysis.

(a) Load all CSV files containing transacted flats in a given `data` directory and merge all of them into a single Pandas DataFrame. Drop the `remaining_lease` column from the merged DataFrame. Are there any columns that contain null values or empty strings?

(b) Convert the `month` column to the date-time format. Design a visualization to analyze the `month` column by considering it as a numeric date-time and share insights.

(c) The column `storey_range` is in the format “lower TO upper” (e.g. 1 TO 3). Compute a new column called `storey_level` by calculating the average of the lower and upper story values. Drop the `storey_range` column from the DataFrame.

(d) Identify inconsistent `flat_model` and `flat_type` values and perform the standardization of the values.

(e) To perform the following visualizations:
(i). Plot a histogram of the `resale_price` to understand its distribution. Is it normally distributed or skewed?
(ii). Generate a boxplot for the `floor_area_sqm` column. Are there any values that lie outside the expected range? If outliers are present, please provide an explanation for their occurrence.

(f) Design and identify FIVE (5) factors that influence the resale price and offer a rationale for each of these correlations.