The concept of Data products has been around for some time. Obviously, they help in giving agility to end users by putting curated data right into their hands enabling quick insights and aiding quick decision making. Once a business need has been identified and the right product has been curated for that need, it can continue to serve that need for as long as other dynamics don’t change – thus helps improve reusability.
Putting data products on the enterprise data marketplace aids the amazonification of data across that enterprise. It becomes that much easier to make data discoverable and accessible and thus truly empowers the end user by enabling data democratization. It also makes it easy for the organization to monetize its data, by selling it to internal or external customers
So that’s what data products are – they are tangible items (mostly as views or database tables) emanating from a lake or a warehouse, resulting from a process including, but not limited to the below activities
- Understanding specific needs of end users for their needs of forecasting, analytics, trend analysis, reporting etc
- Triaging the needs from different business users
- Prioritizing the critical ones
- Understanding which are the attributes coming in from which sources would be needed to serve these needs
- Development activities for sourcing data components to create these data products
- Putting in place data governance keeping in mind needs of the business users
- Access provisioning
In order to visualize this better, let us look at some examples of data products
- A RAG dashboard of KPIs
- Running comparison of Q-o-Q sales
- Faster route available notification on Google Maps
Now that we are clear on what are data products and how they can be useful to the enterprise, let us now look at data as a product. I have seen that these two terms are often used interchangeably even though there are subtle differences.
Data as a product is a concept or an approach to treating data. Data has always been used for reporting and analytics purposes, and even though the idea was always to eventually improve bottomlines and toplines by putting it to use, data itself was not perceived as a commodity which could be branded, packaged and monetized. This realization is now hitting our customers.
To draw an analogy: when plastic was first discovered, everybody knew that it possessed great potential and was definitely valuable. But it only much later when plastic based products began to be manufactured and its utility value was felt in literally every sphere of life, that the true value of plastic was realized.
Data as a product approach led to the creation of a situation where data on its own strength acquired monetary value. The approach led to the creation of entire industries. Life Science companies often buy data from various sources to enhance their research, drug discovery and product development processes.
Netflix collects data on viewing habits from its subscribers to improve its recommendation engine. Starbucks uses data from its loyalty card program and mobile app to analyze customer purchase behaviour.
If an organization needs to create a data product which can be consumed as is by an internal or external user, obviously, it needs to be of high quality, there shouldn’t be an alternate view which conflicts the truth brought out by this one (single source of truth), it needs to be governed well, lineage and traceability of its attributes has to be spot on, latency has to be optimal, and assembly line observability has to be of the highest magnitude.
While the importance of all these supporting functions was always well known, the apprehension that the final product may be rejected due to one or more issues along the data pipeline makes enterprises to lay more emphasis on these supporting functions and thus drives up the commitment to output a great data product. Netflix knows that it can potentially lose customers if its recommendation engine malfunctions
In summary, data product is a tangible commodity built for a specific business use and data as a product is a org-wide mindset for the creation of a good data product.
