Lead Data Engineer Job at WorkHQ, Los Angeles, CA

NnpDQjZEQkFiZHAxTkdKQTdmZEF5S1ZkOXc9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Shift work,

Similar Jobs

Caring for Family of Companies

Private In-Home Caregiver Job at Caring for Family of Companies

 ...Instead of feeling burnt out and understaffed in an Assisted Living or Care Facility, you'll experience the joy of providing one-on-one care,...  ...2 minutes about how were raising the bar for our amazing in home Caregivers at: Why Join Us Now? At Caring for Family of... 

Impact Fire

Fire Alarm Inspector Job at Impact Fire

Established in 2009, Impact Fire Services was the vision of a group of experienced industry professionals that wanted to build a best-in-class fire protection and life safety company. Through superior customer service and technical expertise, we have developed long-standing... 

PriorityPet Urgent Care of Watkinsville

Experienced Veterinary Assistant Job at PriorityPet Urgent Care of Watkinsville

 ...to hire the most compassionate, experienced, and qualified veterinary assistants to ensure the highest quality of medicine is provided. Our...  ...Qualifications Were looking for: A minimum of 2 years experience as a veterinary assistant or technician A high work... 

Rockydale Quarries Corporation

Quarry Utility Worker Job at Rockydale Quarries Corporation

$1,500 Sign-on Bonus!Rockydale Quarries Corporation, an established aggregate producer, is looking for a Utility worker for its Starkey Road (Roanoke, Virginia) location. Must possess the ability to operate skid loaders, heavy duty off-road earth or rock moving equipment... 

European Wax Center

Wax Specialist Licensed Esthetician or Cosmetologist Job at European Wax Center

 ...outgoing? Does the idea of earning excellent commission drive you? If so, you need to join our team! Qualifications License: Esthetician/Facialist or Cosmetologist License (Required) Benefits Health insurance Dental insurance Matching 401(k) Life...