Software Engineer, Big Data
The Video Engineering Team in the New York office of AppNexus is looking for a software engineer to work on big data/reporting projects. The AppNexus Platform processes tens of billions of log records a day (more than 150TB of data) generated by our real-time bidding engines and ad serving platform. This information is used to generate a variety of aggregated analytics reports which are used by our customers to analyze performance of their ad campaigns and to make informed strategy decisions.
The engineer will work on all aspects of AppNexus’ data pipeline: Data ingestion, validation, aggregation, and presentation. The engineer will work on site, in AppNexus’ New York office, in close collaboration with the engineers from the video team, as well as with members of other teams. The ideal candidate will have interest in big data and reporting applications, and will have extensive knowledge of the required technologies.
About the job:
- Design and develop big data processing routines in Java, building reliable, scalable and testable enterprise-level data processing systems.
- Collaborate with a cross-functional team with diverse technology backgrounds.
- Work closely with product stakeholders and users to understand data and reporting requirements.
- Participate in all stages of the SDLC, from design and development, to deployment and maintenance.
- Perform in-depth analysis and optimization of data pipeline components to ensure smooth execution within strict time and resource limitations.
About your skills and experience:
- Understanding of distributed data processing methodologies and frameworks.Hands-on experience writing MapReduce and Spark jobs in a high load production environment.
- Solid understanding of databases with solid working knowledge of SQL.
- Fluent in Java, with experience in at least one of the following:Python, PHP, NodeJS.
- Experience with reporting/BI systems and/or data visualization tools and techniques is a strong plus.
- Knowledge of Protobuf and Apache Parquet is a plus.
- Good communication skills and ability to work cross-functionally.
- Knowledge of the online advertising business is a strong plus.
More about you:
- You are passionate about a culture of learning and teaching. You love challenging yourself to constantly improve, and sharing your knowledge to empower others
- You like to take risks when looking for novel solutions to complex problems. If faced with roadblocks, you continue to reach higher to make greatness happen
- You care about solving big, systemic problems. You look beyond the surface to understand root causes so that you can build long-term solutions for the whole ecosystem
- You believe in not only serving customers, but also empowering them by providing knowledge and tools