Big Data Management Unlocks Big Insights

Centralized data lake empowers data-driven decision making

Big Data Management Unlocks Big Insights

Story Snapshot

A global telecommunications company needed a central repository for its data after acquisitions led to several separate databases that were costly and time consulting to search.

Results At-A-Glance

  • Increased customer retention
  • Data scientists can identify a customer need 300% faster than previously
  • Moved from batch to real-time updates
  • Increased data accuracy
  • Enables identify more complex challenges and take meaningful action
  • Can now anticipate issues and be proactive and identify areas of cost savings based on call frequency

Key Services

  • Cross-functional stakeholder alignment
  • Data pipeline deployment
  • Long-term data strategy

The Challenge: Time-Consuming Data Analysis

A global telecommunications firm found itself analyzing multiple sets of data across the organization, which was time consuming and costly. The company needed to consolidate several major databases which contained disparate, complex data, resulting from the acquisition of several, smaller firms each of whom used a different system.

The Solution: A Unified Data Lake

The solution consisted of standing up a data lake, a storage repository that holds a vast amount of data. This data would be stored in its raw format, meaning it would look exactly the same way as it does in the source - whether the source of data is a file or a database. A proof-of-concept data lake was stood up in a development cloud environment with sample data to demonstrate how the architecture would function.

Our Process: HCDAgile

1904labs leveraged the Human-Centered Design practice to understand the reality of the client's world. The disparate data, various political verticals and lack of definition around their data science practice, yielded many opportunities for defining, aligning and developing a sustainable solution.

Outcomes: Enabling Data-Driven Decision Making

With the cloud being a few years out, a data lake was built on existing servers. Utilizing Nifi, 1904labs created processors that can accept data inputs from many types of sources and will automatically land data within the data lake, while running a reusable ingestion flow that standardizes some columns such as date, enables PII encryption and handles errors where/as needed. An initial user interface was created that allows a user to upload data from a file with the click of a few buttons and enables the user to monitor progress in the form of provenance data in Kibana and a custom-built data profiler. As data started populating in the lake, the team of data scientists began analyzing and understanding what more they need to become a successful organization for the company. With that, we continued to stay close to their findings and constantly planned for future capabilities. We built an efficient way for the data governance team to provide metadata (definition of a table and its columns) for the data sets. 1904labs automated the creation of a derived data set and maintain a raw data and clean data layer - allowing for the jump back to raw (source) data any time.

As the newly founded data science teams takes on its first few use cases, 1904labs will be there to continue to support their needs and work towards allowing them to quickly and efficiently take data, turn it into information, and with those insights, help drive business decisions and actions.

  • Helped a large enterprise data science practice mature
  • Rapid acquisition and integration of customer-related data across the organization
  • Enabled faster delivery of insights to data customers
  • Enabled the transition from reactive to proactive issue mitigation
  • Efficient ingestion process allowing for rapid data availability from customer-facing functions
  • Allow data scientists to access data quickly and efficiently enabling them to proactively draw insights that can save the business money/time/customer satisfaction.
“We began with the end in mind: helping our client become a data-driven enterprise. A fully automated, user interactive, transparent framework allows for easy access to data from multiple sources and facilitates critical decision-making.”

Chris Lundeberg, 1904labs

Technologies

Ready to write your own success story?

1904labs has the expertise and experience to tackle your toughest problems.

Request a strategy session