Study on Household Consumption and Inflation data

Deriving inferences from datasets sourced from the national data archive.

Repository Structure

Current Datasets

1. Household Consumption Dataset

   • Dataset Link
   • We have taken 2023-24 data for our analysis.
   • Can be accessed through Nesstar Publisher

2. Consumer Price Index and Inflation Data

   • Dataset Link

To get the cleaned dataset: Download the S3-Data and explore it.

You can access the cleaning script from here:

Youtube video: Link

Pipeline overview:

Our final database schema looks like this:

In order to deploy on AWS:

1. Create terraform.tfvars with appropriate access keys to your account(must have a proper IAM role)
2. First deploy GOIStats/bootstrap with the following commands:

terraform init

If credentials problem comes up then configure your environment using:

aws configure

Finally use:

terraform plan

terraform apply -auto-approve

• Upload the psycopg2-layer.zip as a layer in the Lambda function, this packaged layer has been prepared in AWS-Linux OS. (using a zipped package from Windows gives an error.)

Note: Additional charges may apply if large datasets are transferred within the vpc using lambda, we avoided using AWS Glue due to cost constraints.

PowerBI dashboard: