Cloud Data Engineer -
As a data engineer at Agiliz you are responsible to deliver end-to-end cloud data solutions for the client. To enable the client to make data-driven decisions based on my solution of collecting, transforming and publishing data.
To develop those cloud data solutions, you can design, build, operationalize, secure and monitor data processing systems with an emphasis on security, compliance and scalability.
- Eager to work as a team
- Ready to take on a challenge
- Curious for new technologies
- Fond of data
- Familiar with AWS, Google Cloud Platform or Azure
- Building ETL data pipelines
- Automation of builds, deployments, testing and configuration
- Knowledge of programming languages like Java/Scala and Python
- Knowledge of Apache Spark or Beam
- Knowledge of container orchestration
- Knowledge of orchestration tools
- Knowledge of storage systems
What's in it for you?
- Challenging projects with a variety of technologies
- Continuous (peer) learning
- Possibility to participate on a tactical level within out company
- Open and informal company culture
- Great colleagues
Meet your future colleague Casper
For my current project, I work on the Google Cloud Platform. Here I connect data from different data sources like Salesforce, MariaDB, etc. to bundle them in a modernized data warehouse. I build custom Spark ETL pipelines with DataFusion. With the Data Loss Prevention API, I encrypt sensitive data with keys to ensure security and privacy. To orchestrate these data pipelines, I use Cloud Composer (Apache Airflow). Event-triggered functionality is handled by serverless Cloud Functions and object storage in Cloud Storage. I also make technical dashboards that monitor the components and their costs in Data Studio. In the end, the data is stored in a Kimball data warehouse model in BigQuery where the data is consumed by MicroStrategy reports.
In my downtime I like to go to the movies and go cycling to train for the company’s yearly bike event. We are going to Italy for the Duchennes Heroes event, 3 days of cycling with my colleagues while supporting a good cause.