As a data engineer at Agiliz you are responsible to deliver end-to-end cloud data solutions for the client. You are also responsible for enabling the client to make data-driven decisions based on my solution of collecting, transforming and publishing data. Furthermore, you have t be able to develop those cloud data solutions, you can design, build, operationalize, secure and monitor data processing systems with an emphasis on security, compliance and scalability.

For my current project, I work on the Google Cloud Platform. Here I connect data from different data sources like Salesforce, MariaDB, etc. to bundle them in a modernized data warehouse. I build custom Spark ETL pipelines with DataFusion. With the Data Loss Prevention API, I encrypt sensitive data with keys to ensure security and privacy. To orchestrate these data pipelines, I use Cloud Composer (Apache Airflow). Event-triggered functionality is handled by serverless Cloud Functions and object storage in Cloud Storage. I also make technical dashboards that monitor the components and their costs in Data Studio. In the end, the data is stored in a Kimball data warehouse model in BigQuery where the data is consumed by MicroStrategy reports. 

In my downtime I like to go to the movies and go cycling to train for the company’s yearly bike event. We are going to Italy for the Duchennes Heroes event, 3 days of cycling with my colleagues while supporting a good cause.