Responsibilities
API Integration & Data Extraction
Lead the integration of Fenergo APIs to extract relevant KYC and AML data, ensuring seamless connectivity and data flow between systems Design, develop, and maintain scalable data pipelines and ETL processes to support data ingestion from various sources, including databases, APIs, and flat files Ensure robust data extraction processes that maintain data quality and compliance with regulatory requirements Data Processing & Pipeline Development
Utilize Databricks and Apache Spark to design and implement robust data processing pipelines, ensuring high data quality and performance Work with DataFrames for transforming data and implementing the Medallion Architecture Execute SQL queries for data extraction, manipulation, and complex data operations Join datasets and add fields to reports to provide comprehensive analytical insights