Below is an article originally written by PowerToFly Partner Bloomberg, and published on September 20, 2018. Go to Bloomberg's page on PowerToFly to see their open positions and learn more.
Emerging technologies like machine learning and deep learning are key to helping Bloomberg engineers solve complex problems. While working with over 100 million financial instruments, and thousands of distinct attributes associated with each, it's essential that technologists keep pace with the sophistication of the technology, ensuring the data's accuracy and accessibility while leaving room for creativity and innovation.
Engineers at Bloomberg are encouraged to expand their skill sets through projects they undertake. We talked with Gillian Chin, a data scientist in Bloomberg's Office of the CTO, and software developers Rupali Saboo and Sheryl Zhang, from the Data Services team in New York. We discussed their approach to new technologies, the projects they are able to apply them to, and how their work personally helps ensure the accuracy and accessibility of Bloomberg's data, while also providing a superior user experience.
Using Deep Learning to Detect Anomalies in Text Data Sets
As developers on the Data Services team, Sheryl Zhang and Rupali Saboo are responsible for building a machine learning-based anomaly detection platform. While statistical and rules-based checks work well for some classes of problems, they realized they would need a different approach to effectively tackle text data.
"Rule-based checks to detect anomalies would create an unmanageable system, given the variety of errors possible in text data," said Rupali. "Our goal was to create a low-cost and easy to use platform which would work on different text-based datasets. If we could pull this off, the system could be used effectively to detect anomalies in any text datasets. We realized that deep learning was the best way to achieve this."
Rupali Saboo(Photographer: Lori Hoffman/Bloomberg)
"We doubted ourselves and had concerns about the idea at first. Neither of us had any formal deep learning background. We had only been working for a year and this wasn't even in the business plans. We were not sure how to proceed," said Sheryl.
Speaking up proved to be a critical part of the process. "We brought this idea forward to our manager, and were surprised with the wholehearted support we received," Rupali adds. "We got started studying and expanding our knowledge of ML-based anomaly detection methods."
At Bloomberg, continued learning is an essential part of the company culture. Rupali and Sheryl took advantage of their access to world-class on-site training sessions and internal education portals, as well as open courses like Bloomberg's Foundations of Machine Learning. The team also huddled together in weekly ML reading groups and participated in hackathons to sharpen their skills. In a few short months, through late nights and hard work, the team had a Minimum Viable Product ready to go.
Sheryl and Rupali had a strong support network in the Bloomberg Women in Technology (BWIT) organization, which provides mentoring, access to allies and sponsors across the company. BWIT also gave Sheryl and Rupali opportunities to pitch their product to senior management. "BWIT mentors helped us gain confidence, improve our presentation skills and become more entrepreneurial," Rupali said. "Presenting at the BWIT Tech Expo in our Princeton office helped us polish our pitch by forcing us to think beyond the technology and focus instead on the value our product offers."
"The exposure we gained at the Tech Expo led to multiple teams partnering with us on their datasets," Sheryl said. "This experience helped us refine our algorithm and improve its accuracy."
Sheryl ZhangPhotographer: Lori Hoffman/Bloomberg
The system is now running on 5 datasets across different applications, each containing millions of records. It has helped detect and resolve over 10,000 cases where text in the database looked suspicious and was potentially mistyped.
"If we had hesitated and waited until we knew everything, nothing would have gotten done," Sheryl said. "I'm glad we jumped in and made it happen."
The diversity of experiences, backgrounds, and perspectives on Sheryl and Rupali's team, combined with the feedback they solicited from engineering, data and product stakeholders, made creating the deep learning-based platform possible.
"Faced with the difficult problem of text anomaly detection, we engaged in cutting-edge research that benefits every Bloomberg data consumer," said Rupali.
"Data is a key part of our business. It feels great that we took what was an abstract idea and delivered an impactful product," added Sheryl.
Sheryl and Rupali will be presenting their project, "Deep Learning Based Text Anomaly Detection," during the Grace Hopper Celebration 2018 Poster Session, on Wednesday, September 26th, from 11:30 AM-2 PM CDT.
Using Data Science to Improve Bloomberg Clients' Search Experience
In Bloomberg's Office of the CTO, Gillian Chin works on strategic technology initiatives for the company. She is currently focused on the Smarter Autocomplete project, which will improve discoverability in the Bloomberg terminal so that users can find information more quickly. "I can observe the issues our clients are having with search and information discovery on the terminal and I'm determined to improve our clients' experience," she said.
Gillian has a few different roles within the project. She manages the roadmap and sprint work, and conducts the quantitative analysis for assessing user behavior, product adoption and product success. As a data scientist, she traces users' perspectives through observational data to understand how they experience a product wherever possible. Then, she uses these insights to guide future enhancements.
Gillian Chin Photographer: Lori Hoffman/Bloomberg
"Right now, we're slowly evolving the way the command line and autocomplete can help users discover information," said Gillian. "The current process is rigid in that it is built to support the workflow of: load a security, run a function." Gillian's innovative work will allow users to interact with the command line in a more intuitive fashion, so that the discovery paths that users traverse become increasingly more fluid and accessible.
In addition to improving the company's systems, Chin, who holds a Ph.D. from Northwestern University, also chaired Bloomberg's 2018 Data Science Research Grant and 2018 Data Science Ph.D. Fellowship programs. "These programs bring visibility to Bloomberg as a powerhouse of research in the fields of data science and machine learning, and demonstrate the company's willingness to engage and invest in solving important business and technological problems," she said. "It also connects Bloomberg with academics working on cutting-edge research and tangibly supports those communities."
While Bloomberg provides technologists like Gillian with broad parameters that encourage creativity, "I also appreciate how the company operates and how philanthropic Bloomberg is," she said. "I feel the imperative to work at a company with values I strongly believe in, and Bloomberg definitely is one."
Gillian is participating in the "Unexpected Yet Common: Industry after Ph.D." career panel at Grace Hopper on Friday, September 28th, from 1-2 PM CDT (Lanier J in the Hilton Americas-Houston).
