Data Platform Engineer

Job description

Job responsibilities

• Build the data backbone for autonomous factories, transforming machine signals, quality events, work orders, and application changes into trusted data used by scheduling, ML, and operations.
• Build ingestion, CDC, and streaming capabilities for transactional data, events, telemetry, and files; explicitly manage ordering, deletes, retries, replay, idempotence, and backpressure.
• Define versioned data and event contracts with upstream teams, supported by testing and service targets for freshness, completeness, and correctness.
• Model telemetry, quality events, work orders, and operational data into datasets with explicit grain, identity, time, provenance, and history.
• Own Dagster orchestration, dbt transformation, data CI/CD, backfills, lineage, observability, and offline feature datasets for ML.
• Collaborate with Manufacturing Operations and Infrastructure to acquire data from machines, PLCs, historians, OPC-UA, MTConnect, and MQTT sources as needed.

Job requirements

• Experience building and operating production data infrastructure or distributed data systems, including on-call ownership and recovery efforts.
• Strong production Python and advanced SQL and data-modeling skills, including incremental processing, temporal data, and schema evolution.
• Experience with Kafka or another event-streaming platform, plus CDC or other stateful incremental pipelines.
• Experience operating Snowflake, and with a lakehouse table format such as Iceberg, Delta, or Hudi, including expertise in partitioning and compaction.
• Experience with tools such as Dagster, Airflow, Argo, or Prefect; dbt or similar transformation frameworks; and Kubernetes or infrastructure as code.
• Strong judgment regarding contracts, failure modes, and the needs of downstream analytics, ML, and operational systems.