Big Data/PySpark Engineer

Posted 9 days ago
Main Location
Seattle, WA, United States
Open jobs

ns and design-led experiences for its clients, delivered through the power of people and the Microsoft ecosystem. Our professionals combine technology, business, and industry expertise to build and deploy solutions to realize results for clients and their customers. Avanade has 34,000 digitally connected people across 24 countries, bringing clients the best thinking through a collaborative culture that honors diversity and reflects the communities in which we operate. We welcome all and seek talented individuals who can bring their whole self to work, build inclusive teams and encourage diversity inside and outside the organization. Majority owned by Accenture, Avanade was founded in 2000 by Accenture LLP and Microsoft Corporation.

How we support you:

We believe in gender equity and an inclusive community. We offer a comprehensive benefits package: generous vacation allowance disability coverage, retirement plans, paid maternity and paternity leave, life insurance, hotel and travel discounts, extended benefits to cover items that support your well-being, health, dental and vision insurance, professional development and paid Microsoft certification opportunities.

About you:

You draw on your considerable experience in bringing data and statistics to life to solve sometimes complex problems, and you’re comfortable looking after several projects at once. You’re able to make your own decisions while at the same time supporting more junior team members. As a Big Data/PySpark Engineer at Avanade, you will have a deep understanding of the architecture, performance characteristics and limitations of modern storage and computational frameworks, with experience implementing solutions that leverage: HDFS/Hive; Spark/MLlib; Kafka, etc. You will have knowledge in Apache Spark and/or Python programming, deep experience in developing data processing using PySpark such as reading data from external sources, merge data, perform data enrichment and load into target data destinations. Deep experience in developing data processing tasks using PySpark such as reading data from external sources, merge data, perform data enrichment and load into target data destinations. Knowledge of python packaging, azure/data lake, and data bricks. Experience with ML. Manage large volumes of structured and unstructured data and extract & clean data to make it amenable for analysis. 80% travel is required.

Key Technologies:

Must-haves:

Apache Spark

Python

PySpark

Hadoop

SQL

Azure or AWS

Pluses:

Cosmos DB

TensorFlow

Apache airflow

Snowflake

Linux OS

.

Key Role Responsibilities:

Demonstrated expertise working with and maintaining open-source data analysis platforms, including but not limited to:

Pandas, Scikit-Learn, Matplotlib, TensorFlow, Jupyter and other Python data tools

Spark (PySpark), HDFS, Kafka and other high-volume data tools

SQL and NoSQL storage tools, such as MySQL, Postgres, Cassandra, MongoDB and ElasticSearch.

Deep understanding of the architecture, performance characteristics and limitations of modern storage and computational frameworks, with experience implementing solutions that leverage: HDFS/Hive; Spark/MLlib; Kafka, etc.

Technical background in computer science, data science, machine learning, artificial intelligence, statistics or other quantitative and computational science

Hands-on experience using Big Data and statistical analysis tools such as Hadoop/Spark, SQL

Experience transforming Data at a scale using Spark/PySpark

Working experience with Linux OS (Redhat/Ubuntu)

Excellent communication skills (speaking, presenting)

Preferred Years of Work Experience:

You likely have about 3-6 + years of relevant professional experience.


Help us maintain the quality of jobs posted on PowerToFly. Let us know if this job is closed.
Mission
We're a community of women leveraging our connections into top companies to help underrepresented women get the roles they've always deserved. Simultaneously, we work to build truly inclusive hiring processes and environments where women can thrive and not just survive.
Are you hiring? Join our platform for diversifiying your team
Big Data/PySpark Engineer
Avanade Inc.