Middle Big Data Engineer
$500 Reward
$500 Reward
Overview
Our client stands as a trailblazer in the digital subscription space, offering a rich tapestry of ebooks and audiobooks to a global audience. At the heart of its mission is the democratization of information and the creation of a platform where the exchange of ideas, stories, and knowledge transcends traditional barriers. With a commitment to innovation and excellence, the company has carved out a niche for itself by redefining the digital reading experience and fostering a vibrant community of writers, readers, and learners. Operating in a dynamic and ever-evolving industry, our client offers an exciting opportunity for growth, learning, and significant impact.
Challenges you would face
- Craft and manage end-to-end data pipelines for large, complex datasets
- Ensure data quality and integrity
- Collaborate with teams across functions like Data Science or Content
- Engineering to enhance business-critical pipelines
- Develop tools for reliable data consumption by downstream customers
- Handle Spark jobs optimization, with a focus on Spark version 3.1 and 3.3
- Address infrastructure challenges, particularly around Airflow
The ideal candidate
- Minimum 3 years in data engineering, managing complex data pipelines
- Proficient in Spark/Spark Streaming
- Skilled in Scala and/or Python, with a high proficiency level required in at least one
- Familiar with SQL
- Upper-intermediate level of English
Plus
- Experience with Kafka or similar streaming platforms
- Familiarity with Terraform, Airflow
- Understanding of AWS data services
- Proficient with Jira, Slack, JetBrains IDEs, Git, Docker, Jenkins
- Experience with DataBricks and Infrastructure as Code (Terraform)
What the company offers
- Remote work flexibility
- Engagement in high-impact projects with global reach
- Opportunities for professional growth and development within a supportive and innovative environment
This job is no longer open, to check out our open roles please go to