Associate Data Engineer, Enterprise Data Engineering
Overview
Design and build scalable ingestion and processing pipelines for unstructured content, and develop reusable patterns and accelerators for common data processing use cases.
Key Responsibilities
- Design and build scalable ingestion and processing pipelines for unstructured content including PDFs, audio, video, and images.
- Implement document processing workflows covering OCR, metadata extraction, content enrichment, chunking strategies, and vector indexing to support RAG solutions.
- Develop multimodal processing capabilities that handle diverse source formats consistently and at scale.
- Build and maintain embedding, retrieval, and re-ranking pipelines integrated with enterprise search and AI platforms.
- Develop a library of reusable pipeline templates, blueprints, and starter kits for common unstructured data processing use cases.
- Build Infrastructure as Code (IaC) patterns and standardized deployment configurations that domain teams can adopt independently.
- Document reference architectures, cookbooks, and implementation guides to reduce time-to-value across teams.
- Partner with data architects, AI engineers, and business teams to translate requirements into reliable data products.
- Contribute to code reviews, engineering standards, and shared documentation practices.
Required Experience
5-10 years of experience in data platform operations, cloud operations, SRE, DevOps, DataOps, production support, or platform engineering disciplines. Demonstrated experience leading engineers or technical teams in a hands-on capacity. Experience managing or co-leading vendor relationships. Proven experience in data engineering with a focus on unstructured data processing and AI/search use cases. Experience building RAG or enterprise search pipelines including chunking, embedding, and indexing workflows. Familiarity with multimodal content processing (PDF, audio, video) and associated tooling. Understanding of DevOps practices: CI/CD, automated testing, and pipeline monitoring.
Qualifications
Typically requires a Master's degree with 5 years of experience or a Bachelors Degree with a minimum of 7 years of relevant experience, or equivalent combination of education and experience. SAFe or other relevant Agile certifications. Databricks Certified Data Engineer Associate or equivalent.