Responsibilities:

• Develop backend systems that interact with distributed Big Data platforms such as Databricks, and Apache Spark.

• Build microservices and APIs tailored for Big Data operations.

• Create APIs to expose processed data insights to front-end applications and external systems.

• Integrate APIs with other systems to facilitate data sharing and analytics.

• Implement data ingestion pipelines and transformation workflows for structured and unstructured data.

• Optimize backend systems for high throughput and low latency, ensuring data processing performance at scale.

• Manage schema design and query optimization for massive datasets.

• Work closely with data engineers, data scientists, and frontend teams to align backend infrastructure with business needs.

• Participate in the design of end-to-end solutions leveraging Big Data technologies.

• Analyze problems and find resolution, as well as work closely with the administration to monitor performance and advise any necessary infrastructure changes.

Qualifications:

• At Least 4 years of experience as a Software Developer or Data Engineer

• Strong coding skills i.e. JavaScript, Node.js, SQL, Python, pySpark

• Experience working with various forms of data infrastructure inclusive relational databases such as SQL, Delta Lake, and Spark

• Experience with orchestration tools e.g. batch and real-time data processing

• Knowledge of cloud-based Big Data services such as Databricks, Azure Data Lake, or AWS EMR.

• Strong problem-solving and analytical abilities for handling complex data challenges.

• Effective communication and teamwork to bridge technical and business needs.

• Continuous learning to adapt to evolving Big Data technologies.