Role Summary:

As a Data Engineer, you will contribute to designing, maintaining, and enhancing various data services and infrastructure. You'll work with cross-functional teams to ensure seamless data flow for critical decision-making processes.

Key Activities:

• Data Infrastructure Design and Maintenance: Support the architecture, maintenance, and enhancement of analytical and operational services and infrastructure, including data lakes, databases, data pipelines, and metadata repositories.
• Collaboration: Assist in designing and implementing data schemas and models, integrating new data sources, and collaborating with other data engineers to implement cutting-edge technologies.
• Data Processing: Develop and optimize data processing systems to support the organization's growth and improvement initiatives.
• Workflow Management: Use workflow scheduling and monitoring tools like Apache Airflow to ensure efficient data processing and management.
• Quality Assurance: Implement testing strategies to ensure the reliability and usability of data processing systems.
• Continuous Improvement: Stay updated on emerging technologies and best practices in data engineering and propose optimizations.

Required Skills:

• Technical Expertise: Proficient in Unix environments, cloud computing (GCP), Python frameworks (e.g., pandas, pyspark), version control systems (e.g., git), and workflow scheduling tools (e.g., Apache Airflow).
• Database Proficiency: Experience with columnar and big data databases like Athena, Redshift, and Hive/Hadoop.
• Containerization: Experience with container management tools like Docker and Kubernetes.
• CI/CD: Knowledge of CI/CD tools such as Jenkins or CircleCI.
• Experience: Minimum of 6 years of experience in data engineering or related fields.

Nice-to-have requirements:

• Programming Languages: Familiarity with JVM languages like Java or Scala.
• Database Technologies: Experience with RDBMS (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., DynamoDB).
• BI Tools: Exposure to enterprise BI tools like Tableau or PowerBI.