Data Ingestion & Processing
- Build and maintain ingestion pipelines from APIs, files, and operational systems into our platform.
- Implement robust transformation logic: validation, deduplication, idempotency, enrichment, and reconciliation.
- Design data models and storage strategies for SQL (PostgreSQL/MySQL) and MongoDB, including indexing and query optimization.
- Orchestrate workflows using Airflow (or equivalent): scheduling, dependencies, retries, alerting, backfills.
Backend (Node.js / TypeScript)
- Develop and maintain backend services and APIs using Node.js (preferably TypeScript).
- Build scalable REST APIs (and/or GraphQL if applicable): pagination, filtering, auth, role-based access, and rate limits.
- Implement background processing patterns (queues/workers/cron) for ingestion and heavy jobs.
- Write clean, testable modules with logging, error handling, and performance profiling.
Frontend (React / HTML)
- Build user-facing web UIs using React (and occasionally HTML/CSS pages), focused on operational workflows.
- Implement data-heavy UI components: tables, filters, search, forms, upload/download, status views, and audit trails.
- Integrate frontend with backend APIs with strong UX around loading states, errors, and edge cases.
AWS, Deployment & Operations
- Deploy and operate services on AWS, ensuring reliability, security, and cost awareness.
- Work with AWS services such as S3, IAM, EC2/ECS/EKS/Lambda (as applicable), RDS, and CloudWatch.
- Containerize services using Docker; contribute to CI/CD pipelines and environment promotion practices.
- Monitor production: logs, metrics, alarms, pipeline failures; perform RCA and implement fixes.
Engineering Practices
- Use Git/GitHub for code reviews, branching strategies, PR hygiene, and collaboration.
- Maintain documentation: system diagrams, onboarding notes, runbooks, pipeline/data flow docs.
Collaborate with product, operations, and engineering teams to ship reliable features quickly.