Department Manager, AI - Data Engineer

Responsibilities:

• Develop and maintain robust data pipelines to ingest, process, and transform raw data into formats suitable for LLM training.

• Conduct meeting with users to understand the data requirements and perform database design based on data understanding and requirements with consideration for performance

• Maintain data dictionary, relationship and its interpretation

• Analyze problem and find resolution, as well as work closely with administrators to monitor performance and advise any necessary infrastructure changes

• Work with business domain experts, data scientists and application developers to identify data that is relevant for analysis

• Develop big data solutions for batch processing and near real-time streaming

• Own end-to-end data ETL/ELT process framework from Data Source to Data warehouse

• Select and integrate appropriate tools and frameworks required to provide requested capabilities

• Design and develop BI solutions

• Hands-on development mentality, with a willingness to troubleshoot and solve complex problems

• Keep abreast of new developments in the big data ecosystem and learn new technologies

• Ability to effectively work independently and handle multiple priorities

Qualifications:

• Bachelor degree or higher in Computer Science, Computer Engineering, Information Technology, Management Information System or an IT related field.

• 3+ year's experiences in Data Management or Data Engineer (Retail or E-Commerce business is preferrable)

• Expert experience in query language (SQL), Databrick SQL, PostgreSQL

• Experience in Big Data Technologies like Hadoop, Apache Spark, Databrick

• Experience in Python is a must

• Experience in Generative AI is a must

• Knowledge in machine/statistical learning, data mining is a plus

• Strong analytical, problem solving, communication and interpersonal skills.

• Having good attitude toward team working and willing to work hard.