Pipeline

The pipeline project was a project designed to learn more about the challenges of creating data pipelines.

The system involved creating a system which scraped APIs, transformed the data, stored it in a columnar datastore, and exposed it via a dashboard and UI search tools.

More project docs will be added here in the future.