Senior Data Engineer

At DetailPage.com, we specialize in helping Amazon brands grow their organic traffic through content optimization and market segment share insights. We use a patented (US Patent No. 12,093,340), AI-assisted process to identify high-traffic, relevant keywords and rewrite product content to drive sustainable organic traffic.

We are seeking a highly skilled and motivated Data Engineer to join our team. In this role, you will leverage your expertise in AWS, Python, SQL, and data management in general to build and optimize scalable data pipelines, support our ETL/ELT and API processes, and enable seamless data flow across our organization and to our end users. You will work closely with our engineering, analytics, and product teams to drive data-driven solutions and ensure efficient data infrastructure.

Responsibilities:

▪ Design, develop, and maintain scalable data pipelines using a variety of AWS services

▪ Work with relational databases such as Aurora (PostgreSQL) and Redshift for large-scale data management

▪ Develop and optimize ETL/ELT processes to ensure smooth data integration from multiple sources

▪ Build and maintain serverless applications using AWS services like Lambda, Fargate, and SQS

▪ Collaborate with cross-functional teams to ensure data accessibility and integrity across different platforms

▪ Use Python, SQL, and Pandas for data manipulation, analysis, and automation of workflows

▪ Implement unit testing to ensure code reliability, using PyTest

▪ Manage API integrations and development, using FastAPI

▪ Assist with infrastructure as code (IaC) using CDK

Must Have Skills & Qualifications:

▪ Extensive Work Experience: 5+ years in Data Engineering with AWS and Python

▪ Strong AWS Experience: Particularly with services like Aurora (PostgreSQL or MySQL), Redshift, Fargate, Lambda, SQS, and IAM.  Experience using tools like Boto3 and AWS CLI

▪ Linux Proficiency: Comfortable in a Linux environment, including things like shell scripting, common CLI tools, creating and testing Linux based Docker images

▪ SQL Mastery: Expertise in complex SQL queries and database optimization

▪ Advanced Python: Strong hands-on experience with Python, ideally with strong experience with Data Engineering tools, like Pandas, SQLAlchemy, Pandera

▪ Boto3 Expertise: Solid understanding of AWS SDK for Python (Boto3), AWS CLI, and AWS automation tools

▪ Git Proficiency: Comfortable with version control, branching, and collaboration using Git

▪ ORM Skills: Ideally with SQLAlchemy, for database interactions

▪ ETL/ELT Experience: Proven ability to build and optimize extract, transform, and load (ETL) processes

▪ Unit Testing: Familiarity with unit testing practices, ideally using PyTest or similar tools

Preferred Skills & Qualifications:

▪ Airflow Expertise: Experience with Airflow, particularly AWS Managed Airflow, for scheduling and orchestrating data workflows

▪ API Development: Ideally with FastAPI for building fast, scalable APIs

▪ Infrastructure as Code (IaC): Experience with CDK or similar tools to automate cloud infrastructure provisioning

▪ OpenAI: Some exposure to OpenAI, in particular automating GPT and using embedding vectors

▪ Familiarity with caching tools like Redis.

▪ Basic DevOps knowledge for enhancing CI/CD pipelines and optimizing development workflows.

Why Join Us?

This full-time, remote position offers a fast-paced, collaborative environment where you’ll have the opportunity to build scalable, cutting-edge data infrastructure that powers AI-driven insights for leading Amazon brands. You’ll work with a highly skilled team of engineers and data scientists, tackling complex challenges in data processing, automation, and analytics. If you’re looking for a role where you can innovate, optimize large-scale data pipelines, and shape the future of e-commerce intelligence, we’d love to hear from you!

Please send your resume and a brief introduction to growth@detailpage.com. We look forward to hearing from you!

‍