PLEASE NOTE THIS IS AN EQUITY-ONLY ROLE AND THE INTERVIEWS WILL COMMENCE IN FEBRUARY 2025.
Stealth-Mode Start-Up Client is seeking skilled Data Engineers to design, build, and maintain scalable data pipelines and infrastructure that power advanced analytics, AI, and machine learning initiatives across the platform. This role will focus on data ingestion, storage, processing, and integration, ensuring that data flows efficiently across systems and remains secure, reliable, and readily available for analysis.
The ideal candidate will have expertise in big data technologies, cloud infrastructure, and data architecture design, with a passion for enabling data-driven decisions at scale.
To apply, please provide a CV, your compensation requirements (including salary expectations for when funding is secured) and a cover letter/note that explains why you are interested and how you meet the requirements. Please note that submissions received without all the requested information will be automatically disqualified and rejected.
Key Responsibilities:
- Design and implement scalable ETL/ELT data pipelines to collect, process, and store large datasets from diverse sources.
- Develop and maintain a robust data architecture to support real-time and batch data processing workflows.
- Integrate data from multiple platforms, APIs, and third-party services into centralized data lakes and warehouses.
- Optimize and manage cloud-based data infrastructure (e.g., AWS Redshift, GCP BigQuery, Azure Synapse).
- Implement data validation and integrity checks to ensure data accuracy, completeness, and consistency across systems.
- Build and manage real-time data streaming pipelines using technologies like Apache Kafka, Spark Streaming, or similar.
- Work closely with Data Scientists, Analysts, and Product Managers to understand data requirements and ensure data availability for analytical models.
- Continuously optimize data storage and retrieval processes for performance and cost efficiency.
- Implement data encryption, access control, and compliance with regulatory standards (e.g., GDPR, CCPA).
- Set up monitoring systems and logging mechanisms to track data flow, detect anomalies, and resolve pipeline failures.
- Maintain detailed technical documentation for all data pipelines, workflows, and infrastructure components.
Source ⇲
