PowerToFly
Recent searches
  • Events
  • Companies
  • Resources
  • Log in
    Don’t have an account? Sign up
Filters
Clear All
Advanced filters
Job type
  • Reset Show results
Date posted
  • Reset Show results
Experience level
  • Reset Show results
Company
  • Reset Show results
Skills
  • Reset Show results
Clear All
Cancel Show Results
Active filters:
Results 10518 Jobs
Loading...
Loading more jobs...

No more jobs to load

No more jobs to load

Data Engineer (1 - 2 years of experience in Big Data, Spark, Hadoop, Shell scripting)

VISA

Save Job
VISA

Data Engineer (1 - 2 years of experience in Big Data, Spark, Hadoop, Shell scripting)

Onsite Bengaluru, India Full Time Mid-Level
Posted a day ago
Save Job

Watch this video to learn more about VISA

Job Details

Job Description

At Visa, the Corporate Information Technology, Billing & Incentives Platforms team, enables Visa's revenue growth through flexible pricing engines and global revenue platforms built on next-generation technologies. This includes managing system requirements, evaluating cutting-edge technologies, design, development, integration, quality assurance, implementation and maintenance of corporate revenue applications. The team works closely with business owners of these services to deliver customer developed solutions, as well as implement industry leading packaged software. This team has embarked on a major transformational journey to build and implement best of breed revenue and billing applications to transform our business as well as technology.  

The candidate should enjoy working on diverse technologies and should be excited to take initiatives to solve complex business problems and get the job done while taking on new challenges. You should thrive in team-oriented and fast-paced environments where each team-member is vital to the overall success of the projects.

Key Responsibilities

  • Design, develop, and maintain automated test frameworks and scripts using PySpark for large-scale Big Data applications.

    Collaborate closely with data engineering, development, and DevOps teams to understand data pipelines, workflows, and transformation logic.

    Build and execute robust tests to validate data ingestion, processing, and transformation in Hadoop and Spark ecosystems.

    Perform comprehensive data validation, data integrity, and performance testing across Spark, Hive, and RDBMS data layers.

    Apply deep understanding of Spark concepts (RDDs, DataFrames, Datasets, Spark SQL) to create reusable test automation components.

    Leverage Shell scripting and Unix-based tools for automation, log analysis, and environment configuration.

    Integrate test automation into CI/CD pipelines to ensure continuous testing and high-quality data delivery across environments.

    Identify, document, and track defects, working proactively with developers to ensure quick resolution. Knowledge of Jira Standards.

    Optimize and tune Spark jobs and query performance for scalability and efficiency.

    Maintain detailed documentation, including test plans, test cases, and results, aligned with evolving business requirements.

    Conduct manual and functional testing for critical payment and financial data applications to ensure compliance and accuracy.

    Operate independently in a fast-paced, distributed environment, balancing multiple priorities to meet service-level goals.

    Collaborate effectively across teams and time zones, fostering transparent communication, trust, and long-term partnerships with business stakeholders.

This is a hybrid position. Expectation of days in office will be confirmed by your hiring manager.


Qualifications

Basic Qualifications:
Bachelor's degree, OR 3+ years of relevant work experience

Preferred Qualifications:

  • Bachelor’s degree in computer science, Information Technology, or related field.
  • Relevant certifications in Big Data, Spark, QA.
  • Experience with on premise infrastructure
  • Knowledge of ETL processes and tools.

Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.


Required Skills
  • PySpark
  • big data applications
  • data processing workflows
  • Hadoop ecosystem
  • Spark applications
  • Spark SQL
Company Details
VISA
 Foster City, CA, United States
Work at VISA

At Visa, we are driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid. As our products and... Read more

Did you submit an application for the Data Engineer (1 - 2 years of experience in Big Data, Spark, Hadoop, Shell scripting) on the VISA website?