Data Engineer - Quality Assurance

Responsibilities

• Automate data quality and reconciliation checks across storage layers such as Snowflake, SQL, and RDF/SPARQL databases.
• Test and verify data lineage, governance, and visualization components using Snowflake, data catalogs (e.g., DataHub), Thoughtspot, and other visualization tools.
• Integrate test suites into core infrastructure orchestrated by Apache Airflow, monitor pipeline health, alerting, and observability via Prometheus and Grafana Cloud.
• Establish AI Evaluation Loops (Evals) and Guardrails: build verification protocols, structural tests, checks, and watchdog agents to validate AI-generated artifacts.
• Integrate automated testing workflows into CI/CD pipelines using GitHub Actions, ensuring continuous stability and quality gates across all environments.
• Validate ETL and dbt transformations across Data Lakehouses, rigorously testing data progression through a Medallion Architecture.
• T...