As Big Data Engineer you will be part of the team that shapes
Mango’s data strategy, architecture and infrastructure. You will
develop state of the art data processing solutions to enable use cases
that are very innovative in the retail space, such as geoanalytics,
robotization, omnichannel personalization, the connected store, consumer
360º and real-time decisions.
We expect you to:
- Deploy the latest data processing and streaming technologies such as Databricks Spark, Kafka and AWS ecosystem
- Develop and operate the data lake, data processing and data science platforms end-to-end:
- Design CI/CD pipelines for elements in the cluster like a DataOps
- Develop scalable and reliable data integrations to feed the data science models
- Develop integrations with other key components of the architecture (visualization tools, Jupyter notebooks…)
- Manage and orchestrate the cluster with proper job monitoring and recovery mechanisms
- Work
with a solid engineering approach (e.g., continuous integration,
testing automation, etc) in a strong team of data engineers and data
scientists
- Stay connected to the latest developments in BigData and invest time in bringing new use cases and improvements to Mango
Requirements:
• Degree in Computer Science or related technical field
• At least 3 years of development experience in Python, R, Scala, Javascript, or any other object-oriented language
•
Experience in developing scalable ELT, data integration processes
with Spark, Kafka, Kinesis or any other streaming technology
• Understanding of engineering best practices: write tests, use automation, build continuous integration pipelines, etc
• Experience with BigData datastore design, query and optimization (S3, Databricks, Athena, etc)
• Experience in constructing and maintaining complex high-volume data workloads and orchestrating dependencies
