Data Engineering Manager- Remote- EST work hours

Job Title: Data Engineering Manager

Company Overview

We are a rapidly growing payments platform that provides a full suite of embedded payment and risk management solutions tailored for vertical SaaS companies. Our platform allows SaaS providers to integrate payments and financial services within their applications, creating a new revenue stream in a matter of weeks. We have demonstrated strong product-market fit, leading to exponential growth, and we’re excited to continue this trend.

About the Team

Our team is a dedicated, global group of payments and software experts providing vertical software companies with an all-in-one platform and a personalized approach. We help our clients leverage embedded payments to grow, innovate, and transform. Our customers are top-tier SaaS providers who aim to make their offerings more engaging and increase revenue by adding integrated payment solutions for their users.

About the Role

As the Data Engineering Manager, you will lead the design and architecture of our data platform, supporting financial reporting, reconciliation, forecasting, and data delivery for both internal and external stakeholders. You will guide the evolution of our data infrastructure, focusing on scalability, resilience, and adaptability to customer needs. This role involves overseeing data engineering across multiple teams and fostering best practices for data pipelines in a complex, multi-functional platform.

Key Responsibilities

Technical Leadership

• Provide technical expertise in big data technologies.
• Enhance and optimize our data lake, improving query performance and integrating additional data sources.
• Build ETL pipelines for client-facing data-driven services and internal reporting, including historical analysis, risk assessment, and financial forecasting.
• Develop and communicate data strategies and roadmaps that align with the product portfolio and business goals.
• Define high-level migration plans to bridge current and future state data infrastructure.
• Lead technology environment analysis, identifying areas for improvement and recommending solutions.
• Manage data assets and the data catalog to promote reusability.
• Document data lake architecture, data catalog, and dictionary.
• Oversee access management for the data lake, ensuring compliance with security standards in coordination with InfoSec.
• Maintain ETL code and configurations in version control systems.
• Drive innovation by adopting new technologies to enhance core data assets, including SQL-based, NoSQL, and cloud-based data platforms.
• Identify opportunities to improve platform robustness and scalability.
• Align technical decisions with organizational goals and advocate for the needs of emerging business areas.

Team Leadership

• Provide dotted-line leadership to a distributed team of data engineers, including both full-time and contractor resources, while remaining hands-on.
• Collaborate with and support the business intelligence and data science teams.
• Mentor data engineers and software engineers in data architecture principles and practices.

Project Management

• Prioritize projects in coordination with product management, engineering, operations, and finance.
• Support teams throughout the project lifecycle.
• Fulfill the needs of the business intelligence, financial reporting, and data science teams.

Data Infrastructure

• Develop and support an extended data infrastructure to meet reporting and analysis needs.
• Ensure reliable operations for CDC and ETL pipelines.
• Define, track, and uphold Service Level Objectives (SLOs).
• Manage large and complex data sets to meet business requirements.
• Monitor and manage Data Lake operating expenses, optimizing for cost-efficiency.
• Design and implement operational infrastructure for data pipelines.

Desired Skills and Experience

Technical Skills

• Programming Languages: Python, Java, Scala, C#, SQL
• Data Storage: HDFS, AWS S3
• Data Platforms: Databricks, Snowflake, Redshift
• Data Lakes: Apache Hudi, Delta Lake
• Relational Databases: Postgres, MySQL
• NoSQL Databases: DynamoDB, MongoDB
• Data Processing Engines: Apache Spark, Flink, AWS EMR
• Query Engines: Presto, AWS Athena
• Change Data Capture (CDC): Debezium, Qlik, Fivetran
• Business Intelligence Tools: QuickSight, Tableau, DBT, Power BI
• Streaming Technologies: Kafka, RabbitMQ, ActiveMQ
• Data Governance: AWS Lake Formation
• Data Science/ML: NumPy, SciPy, Pandas, TensorFlow, PyTorch, SageMaker
• Storage Formats: Arrow, Parquet, AVRO
• Serverless Computing: AWS Lambda, Step Functions
• AWS Services: IAM, RAM, EC2

Soft Skills

• Strong communicator, adept at explaining technical concepts to non-technical stakeholders.
• Highly collaborative, working closely with product development teams.
• Excellent problem-solving and adaptability skills, with a forward-thinking mindset for evolving tools and technologies.
• Critical thinker with a proactive approach to risk management and data quality.

Management Skills

• Proven leadership and project management skills.
• Strategic thinker capable of aligning data engineering with business objectives.
• Skilled in managing infrastructure costs and driving efficient use of resources.

Experience

• Bachelor’s degree in computer science, engineering, or a related field.
• 15+ years in data engineering, with extensive experience in end-to-end design and implementation.
• 5+ years of team management in data engineering.
• Strong hands-on experience with AWS cloud platform.
• Background in data lake architecture, governance, and implementation.

Added Bonus

• Experience in fintech, payments, or banking.
• Background in a start-up environment.

What We Offer

A career here offers a chance to be part of the fintech transformation, with opportunities for professional development, collaboration, and contributions to the future of embedded finance. Additional benefits include a competitive salary, comprehensive benefits, and a culture of innovation and growth.