Goal
To future-proof and centralise their data assets, client embarked on a cloud-based data migration and transformation journey.
Role
- Supported the client’s cloud-based data migration and centralization initiative, resulting in a 25% reduction in data access time and saving 000s of pounds in annual maintenance costs by:
- Collaborating on PySpark and SQL-based Azure Synapse notebooks;
- Scripting CI/CD integration for data loading, transformation, validation, and writing into the ‘Strategic Data Platform’ SDP) for HM Courts and Tribunals Service, UK Ministry of Justice.
Tech Stack
SQL ; PySpark ; Azure Data Factory (ADF) ; Synapse Analytics; CI/CD ; Regular Expressions (Regex) ; Git ; Microsoft Azure Cloud
Year
2023