Data Engineering Manager- Remote- EST work hours

Data Engineering Manager- Remote- EST work hours

Job Title: Data Engineering Manager

Company Overview

We are a rapidly growing payments platform that provides a full suite of embedded payment and risk management solutions tailored for vertical SaaS companies. Our platform allows SaaS providers to integrate payments and financial services within their applications, creating a new revenue stream in a matter of weeks. We have demonstrated strong product-market fit, leading to exponential growth, and we’re excited to continue this trend.

About the Team

Our team is a dedicated, global group of payments and software experts providing vertical software companies with an all-in-one platform and a personalized approach. We help our clients leverage embedded payments to grow, innovate, and transform. Our customers are top-tier SaaS providers who aim to make their offerings more engaging and increase revenue by adding integrated payment solutions for their users.

About the Role

As the Data Engineering Manager, you will lead the design and architecture of our data platform, supporting financial reporting, reconciliation, forecasting, and data delivery for both internal and external stakeholders. You will guide the evolution of our data infrastructure, focusing on scalability, resilience, and adaptability to customer needs. This role involves overseeing data engineering across multiple teams and fostering best practices for data pipelines in a complex, multi-functional platform.

Key Responsibilities

Technical Leadership

  • Provide technical expertise in big data technologies.
  • Enhance and optimize our data lake, improving query performance and integrating additional data sources.
  • Build ETL pipelines for client-facing data-driven services and internal reporting, including historical analysis, risk assessment, and financial forecasting.
  • Develop and communicate data strategies and roadmaps that align with the product portfolio and business goals.
  • Define high-level migration plans to bridge current and future state data infrastructure.
  • Lead technology environment analysis, identifying areas for improvement and recommending solutions.
  • Manage data assets and the data catalog to promote reusability.
  • Document data lake architecture, data catalog, and dictionary.
  • Oversee access management for the data lake, ensuring compliance with security standards in coordination with InfoSec.
  • Maintain ETL code and configurations in version control systems.
  • Drive innovation by adopting new technologies to enhance core data assets, including SQL-basedNoSQL, and cloud-based data platforms.
  • Identify opportunities to improve platform robustness and scalability.
  • Align technical decisions with organizational goals and advocate for the needs of emerging business areas.

Team Leadership

  • Provide dotted-line leadership to a distributed team of data engineers, including both full-time and contractor resources, while remaining hands-on.
  • Collaborate with and support the business intelligence and data science teams.
  • Mentor data engineers and software engineers in data architecture principles and practices.

Project Management

  • Prioritize projects in coordination with product management, engineering, operations, and finance.
  • Support teams throughout the project lifecycle.
  • Fulfill the needs of the business intelligence, financial reporting, and data science teams.

Data Infrastructure

  • Develop and support an extended data infrastructure to meet reporting and analysis needs.
  • Ensure reliable operations for CDC and ETL pipelines.
  • Define, track, and uphold Service Level Objectives (SLOs).
  • Manage large and complex data sets to meet business requirements.
  • Monitor and manage Data Lake operating expenses, optimizing for cost-efficiency.
  • Design and implement operational infrastructure for data pipelines.

Desired Skills and Experience

Technical Skills

  • Programming Languages: Python, Java, Scala, C#, SQL
  • Data Storage: HDFS, AWS S3
  • Data Platforms: Databricks, Snowflake, Redshift
  • Data Lakes: Apache Hudi, Delta Lake
  • Relational Databases: Postgres, MySQL
  • NoSQL Databases: DynamoDB, MongoDB
  • Data Processing Engines: Apache Spark, Flink, AWS EMR
  • Query Engines: Presto, AWS Athena
  • Change Data Capture (CDC): Debezium, Qlik, Fivetran
  • Business Intelligence Tools: QuickSight, Tableau, DBT, Power BI
  • Streaming Technologies: Kafka, RabbitMQ, ActiveMQ
  • Data Governance: AWS Lake Formation
  • Data Science/ML: NumPy, SciPy, Pandas, TensorFlow, PyTorch, SageMaker
  • Storage Formats: Arrow, Parquet, AVRO
  • Serverless Computing: AWS Lambda, Step Functions
  • AWS Services: IAM, RAM, EC2

Soft Skills

  • Strong communicator, adept at explaining technical concepts to non-technical stakeholders.
  • Highly collaborative, working closely with product development teams.
  • Excellent problem-solving and adaptability skills, with a forward-thinking mindset for evolving tools and technologies.
  • Critical thinker with a proactive approach to risk management and data quality.

Management Skills

  • Proven leadership and project management skills.
  • Strategic thinker capable of aligning data engineering with business objectives.
  • Skilled in managing infrastructure costs and driving efficient use of resources.

Experience

  • Bachelor’s degree in computer science, engineering, or a related field.
  • 15+ years in data engineering, with extensive experience in end-to-end design and implementation.
  • 5+ years of team management in data engineering.
  • Strong hands-on experience with AWS cloud platform.
  • Background in data lake architecture, governance, and implementation.

Added Bonus

  • Experience in fintech, payments, or banking.
  • Background in a start-up environment.

What We Offer

A career here offers a chance to be part of the fintech transformation, with opportunities for professional development, collaboration, and contributions to the future of embedded finance. Additional benefits include a competitive salary, comprehensive benefits, and a culture of innovation and growth.

Director of Data Architecture

One of our clients is looking for a Director of Data Architecture for a new data initiative they are starting. They are a mid-sized SaaS company that builds a best-in-class software platform for finance and legal professionals to make timely, data-driven informed decisions about investment opportunities.

The Director of Data Architecture will analyze the current data ecosystem (MySQL, PostgreSQL, Mongo, AWS), evaluate new technology, design and architect a new, modern architecture using best-in-class tools and technologies (Hadoop, Snowflake, Spark, data streaming, etc.) as well as staff, lead and manage a team of Data Engineers in an effort to build the new data analytics product suite. All new development, unique opportunity.

This role includes competitive base salary plus an aggressive bonus target, comprehensive benefits, and an opportunity to lead a mission-critical project that will certainly be a career milestone. Please apply to nyjobs@cypressg.com for immediate consideration.

(NYC-based company, remote flexible post-Covid but East-Coast hours are required).

Data Scientist / NLP, ML, DL

My client is a leader in data science and business intelligence solutions that provide intelligent, actionable insights leading to positive results for their customers.  They are currently looking for a Data Scientist for their team located in midtown Manhattan.

Qualified candidates will have 1-2+ years of experience including a strong background in NLP and Machine Learning or Deep Learning, as well as mastery of Python and familiarity with Tensorflow, Scitkit-learn, Pandas, etc. Any knowledge of Apache Spark is a plus.  BS, MS or PhD in CS, Math, or equivalent is required.

The Data Scientist will work with the product team using data science techniques to create data models that can identify actionable concepts and/or trends.  The goal is to map, understand and predict consumer behavior and market conditions.  You will work with other Data Scientists and Computer Scientists in a collaborative team environment, along with various technology and business stakeholders to implement best practices for data management.

The company is a small, dynamic organization that is part of a larger global corporation, so employees get the best of both worlds.  The stability, security and comprehensive benefits of a large company coupled with the spirited, enterprising culture of a startup.  This position includes competitive compensation and lots of room for growth.  Please send a copy of your resume to nyjobs@cypressg.com for immediate consideration.  Local candidates only please.

Senior Data Engineer, Python / Spark

My client is an innovative leader in the cybersecurity industry.  Their systems consume and process huge volumes (multi-terabytes) of data from a variety of sources every day.  They utilize a home-grown Spark based big data analytics platform to execute analytics and data mining on the data sets.  They are currently looking for a talented Data Engineer to work in their Data Team building and scaling the next generation data analytics platform.

As a member of the team you will work in an intimate, collaborative environment along with other Engineers to create systems that organize, analyze, and maintain the data.  The required skills for the position include the following:

  • Python experience (Scala or Java are okay too)
  • Spark, Hive, MapReduce, NoSQL and related technologies (exposure okay)
  • Linux (or Unix)
  • SQL, ETL
  • Amazon AWS  or Google Cloud (could architecture experience with a major cloud provider)
  • Any additional experience with distributed data storeage systems that use parallel process or columnar data stores (Snowflake, Redshift, Cassandra, Druid, HBase, etc.) are all plusses.

The team is looking for the right combination of technical skills, intelligence, curiosity, drive and grit.  They are an innovative startup-like environment that thrives on creative and elegant solutions to complex technical problems.  You have experience improving performance, scalability and availabitity of large systems and have proven industry experience.  You communicate effectively and enjoy collaboration.

This position features a competitive compensation package complete with equity and comprehensive benefits.  If you are qualified and interested, please send a resume to nyjobs@cypressg.com for immediate consideration.

 

This position is conveniently located in midtown and features competitive compensation and a comprehensive benefits package.  Please send a copy of your resume to nyjobs@cypressg.com for immediate consideration.  Local candidates only, please.

Data Engineer, Python / Spark

My client is an innovative leader in the cybersecurity industry.  Their systems consume and process huge volumes (multi-terabytes) of data from a variety of sources every day.  They utilize a home-grown Spark based big data analytics platform to execute analytics and data mining on the data sets.  They are currently looking for a talented Data Engineer to work in their Data Team building and scaling the next generation data analytics platform.

As a member of the team you will work in an intimate, collaborative environment along with other Engineers to create systems that organize, analyze, and maintain the data.  The required skills for the position include the following:

  • Python experience (Scala or Java are okay too)
  • Spark, Hive, MapReduce, NoSQL and related technologies (exposure okay)
  • Linux (or Unix)
  • SQL, ETL
  • Amazon AWS  or Google Cloud (could architecture experience with a major cloud provider)
  • Any additional experience with distributed data storeage systems that use parallel process or columnar data stores (Snowflake, Redshift, Cassandra, Druid, HBase, etc.) are all plusses.

The team is looking for the right combination of technical skills, intelligence, curiosity, drive and grit.  They are an innovative startup-like environment that thrives on creative and elegant solutions to complex technical problems.  You have experience improving performance, scalability and availabitity of large systems and have proven industry experience.  You communicate effectively and enjoy collaboration.

This position features a competitive compensation package complete with equity and comprehensive benefits.  If you are qualified and interested, please send a resume to nyjobs@cypressg.com for immediate consideration.

This position is conveniently located in midtown and features competitive compensation and a comprehensive benefits package.  Please send a copy of your resume to nyjobs@cypressg.com for immediate consideration.  Local candidates only, please.