99 Pyspark Jobs

AVP - Data Engineering  ( Python )

Job Description:

We are seeking a highly skilled, detail-oriented, and motivated Python DQ Automation Developer to join our team. The ideal candidate will be responsible for designing, developing, and maintaining data quality automation solutions using Python. This role requires a deep understanding of data quality principles, proficiency in Python, and experience with data processing and analysis.

Key Responsibilities:

• Design, develop, and implement data quality automation processes and solutions to identify, measure, and improve data quality.
• Write and optimize Python scripts and applications to support data quality initiatives.
• Use Python libraries such as Pandas, NumPy, and PySpark for data manipulation and processing.
• Develop and enhance ETL processes to ensure accurate and timely data integration and transformation.
• Analyze data sets to identify data quality issues, trends, and anomalies, providing insights and recommendations for improvement. Support manual DQ efforts as required.
• Develop and execute test plans to validate the effectiveness of data quality solutions.
• Maintain comprehensive documentation of data quality processes, procedures, and standards.
• Work closely with data analysts, data engineers, DQ testers and other team members and stakeholders to understand data requirements and deliver high-quality data solutions.

Required Skills:

• Proficiency in Python and related libraries (Pandas, NumPy, PySpark, pyTest).
• Experience with data quality tools and frameworks.
• Strong understanding of ETL processes and data integration.
• Familiarity with data governance and data management principles.
• Excellent analytical and problem-solving skills with a keen attention to detail.
• Strong verbal and written communication skills, with the ability to explain technical concepts to non-technical stakeholders.
• Ability to work effectively both independently and as part of a team.

Qualifications:

• Bachelor’s degree in computer science or Information Technology. Advanced degree is a plus.
• Minimum of 7 years of experience in data quality automation and Python Development.
• Proven experience with Python libraries for data processing and analysis.

AVP - Data Engineering  (SQL)

Key Responsibilities:

• Design, develop, and implement data quality automation processes to identify, measure, and improve data quality across the products in the group.
• Write and optimize complex PL/SQL scripts, procedures, and packages to support data quality initiatives.
• Develop and enhance ETL processes to ensure accurate and timely data integration and transformation.
• Maintain Oracle databases, ensuring data integrity, performance optimization, and efficient data storage.
• Analyze data sets to identify data quality issues, trends, and anomalies, providing actionable insights and recommendations for improvement. Support manual DQ efforts as required.
• Develop and execute test plans to validate the effectiveness of data quality solutions, ensuring high standards of data accuracy and consistency.
• Maintain comprehensive documentation of data quality processes, procedures, and standards.
• Work closely with data analysts, data engineers, Manual DQ testers and other team members and stakeholders to understand data requirements and deliver high-quality data solutions.

- Required Skills:

• Proficiency in PL/SQL, SQL, and Oracle database technologies.
• Experience with data quality tools and frameworks.
• Strong understanding of ETL processes and data integration. Talend experience is a big plus.
• Excellent analytical and problem-solving skills with a keen attention to detail.
• Familiarity with data governance and data management principles.
• Strong verbal & written communication skills, with the ability to explain tech. concepts to non-tech. audience.
• Ability to work effectively both independently and as part of a team.

Qualifications:

• Bachelor’s degree in computer science, Information Technology. Advanced degree is a plus.
• Minimum of 8 years of experience in data quality automation and Oracle database management.
• Proven experience with PL/SQL development and optimization.
• Certifications in Oracle, Talend, DQ preferred.

------------------------------------------------------

Job Family Group:

Decision Management

------------------------------------------------------

Job Family:

Data/Information Management

------------------------------------------------------

Time Type:

------------------------------------------------------

Citi is an equal opportunity and affirmative action employer.

Qualified applicants will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.

Citigroup Inc. and its subsidiaries ("Citi”) invite all qualified interested applicants to apply for career opportunities. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View the "EEO is the Law" poster. View the EEO is the Law Supplement.

View the EEO Policy Statement.

View the Pay Transparency Posting