Senior Data Engineer

Responsibilities:

• Responsible for overall data architecture design, and implementation of data platform including Data Lake, Data Lakehouse in the multi-cloud environments. Design and develop the E2E data pipeline: data transformation, ingestion, processing, and surfacing of data for large-scale applications.
• Analyze source data and perform data ingestion in both batch and real-time patterns via various methods; for example: file transfer, API, Data Streaming using Kafka and Spark Streaming. Define and implement the data pipelines for real-time and batch processing across layers in the data platform. Participate in cluster performance tuning to continually improve the quality of data pipeline and overall data platform. Apply software engineering rigor and best practices to data and ML services, including CI/CD, automation, testing, and Infrastructure as Code
• Design data organization structures and taxonomies from multiple data sources for optimal use and enabling data quality and governance. Work collaboratively with the data governance team to maintain data integrity and data availability across data domains
• Work with sale team to support presale activities related to TechX data platform and data engineering solutions. Establish strong collaboration with internal and external stakeholders to ensure project delivery. Ensure appropriate strategy/solution is proposed to customer and monitor actions and results against plans. Provide demonstrations, presentations, training, and consultation for potential clients.

Knowledge & Skills

• Bachelor's degree or equivalent experience in Computer Engineering, Computer Science, Information Technology, or related fields.
• 5+ of Experience in Data Engineer, Data Architecture, Data Modeler, or Data Management with large-scale data in the distributed data platform and overseeing complex data projects in an enterprise environment.
• Excellent proficiency in programing languages (Python, PySpark, SQL, R) and experience with business intelligence solutions (Tableau, PowerBI or Advanced Excel)
• Full stack experience in data collection, aggregation, analysis, visualization, productionisation, and monitoring of ML products – MLOps
• Hand-on experience in Data Lake, Data Lakehouse technology (Spark/Hadoop, Docker or Kubernetes) and cloud platforms such as Azure, AWS, or GCP.
• Hand-on experience in CI/CD practices and tools such as Jenkins, Git, Gitlab, Azure DevOps
• Hand-on experience in Infrastructure as Code such as Terraform or Pulumi.
• Experience in technical consulting, or in a client facing role providing data engineering solutions
• Experience in managing Proof of Concepts and creating solution demonstrations
• Understand business requirements to translate them into technical requirements that technical team members can drive into the project design and delivery
• Understand reports and dashboards requirements, develop data export, data API, or data visualization tools
• Hand-on experience in Agile methodology with with Confluence, Jira, or other similar task management and requirement capturing tools
• Proven ability to quickly and deeply learn new technologies and clearly communicate their value proposition
• Excellent verbal, written, and interpersonal communication skills (both Thai and English)
• Strong organizational & time management skills
• Self-starter with a growth mindset