Data Engineer

Full Time
Main Location
New York City, NY, United States
Open jobs
Data Engineer

We are looking for a Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. You will take ownership of expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for multi-functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up.

In this role you will build and maintain an efficient data pipeline architecture, while assembling large, complex data sets that meet functional / non-functional business requirements.  We are looking for you to Identify, design, and implement internal process improvements, such as automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.  You will build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies.

Principal Responsibilities

  • Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
  • Collaborate with partners from both the business and technology to assist with data-related technical issues and support their data infrastructure needs.
  • Build data and analytical tools for internal customers that assist them in building and optimizing our data organization into an innovative industry leader.
  • Work with data and analytics specialists to strive for greater functionality in our data systems.

Qualifications/Skills Required

  • 5 + years of experience with:
    • Object-oriented/object-function languages: Python, Java, Scala, etc.
    • Hadoop, Spark, Presto, Kafka, etc.
    • Relational SQL and NoSQL databases, including SQL Server, MySQL and Cassandra.
    • Data pipeline and workflow management tools: Airflow, Luigi, etc.
    • AWS cloud services: EC2, EMR, RDS, Redshift, Athena, SQS, etc.
  • Savvy with data science stack (Pandas, NumPy, SciPy)
  • Data Science/Analysis background; Proficient at working with large datasets
  • Unix/Linux command-line experience
  • Experience working with AWS, GCP or Azure
  • Broad understanding of equities, derivatives, futures, FX, or other financial-services instruments

Mission
We're a community of women leveraging our connections into top companies to help underrepresented women get the roles they've always deserved. Simultaneously, we work to build truly inclusive hiring processes and environments where women can thrive and not just survive.
Are you hiring? Join our platform for diversifiying your team
Data Engineer
Millennium Management