Modern Data Platform
Azure Purview, Azure DevOps, Azure ARM, Azure Data Lake Storage
The customer is an American multinational retail corporation with a chain of hypermarkets, departmental discount stores, and grocery stores from the United States.
Data has always been an integral part for the customers business to make intelligent business decisions; however, sprawling data estate across every form of IT technology and a multi-cloud strategy (Azure, Google Cloud, their own datacenter) had increased the complexity of the environment.
They aim to responsibly transform their Sales, Supply Chain and inventory data for the suppliers and consumers into actionable, insights. They had multiple challenges to address, such as quality of data, inability to access data when they need quickly, and not having the robust tools to standardize data management.
- WinWire performed a comprehensive data assessment which included a cross-functional data store that was to bring together a combination of multi-vendor SaaS, Hadoop, Teradata, SAP HANA, Oracle from various divisions ( Core Retail, Real Estate, Finance, HR). One of the most critical components of the data store is the meta-data store and data governance.
- WinWire team has been considered an extension to the Microsoft Engineering team and supported the customer to operationalize data governance solution.
- Developed Apache Atlas API connectors to integrate metadata from Hive,Spark , Sqoop, Teradata, Kafka and ERWin (for business glossary).
The delivered data governance solution comprised of the following key components:
- Data Inventory: The solution to inventory and curate data assets with user-augmented process for validating and resolving any ambiguity in the automated inventory process.
- Data Lineage: Includes impact analysis to identify the impact of a change on any metadata element. Involves storing all the data process and transformations steps that any data element has gone through from its original source to all the endpoints.
- Data Enrichment: Enrichment capabilities through user tagging of content or automatic detection such as personally identifiable data (PID).
- Metadata Exchange: Loading and exporting metadata with third-party tools in a fast, efficient, and accurate manner.
- Security and Privacy: Manage the policies and rules associated with potentially complex privacy rights.
- Custom Development: Flexibility for custom development of components needed ingestion of metadata/ lineage.
- Addressed opportunity areas such as data access, collaboration, security/protection, to create opportunities for data and insight monetization.
- Built a scalable approach to offer better experience to merchants, suppliers, supplier partners.
- Elevated business agility by offering an improved view on improved view on sales, operations, and inventory data.
- Seamless data governance platform that delivered unified experience across very complex set of technologies (legacy, cloud native) and cloud (multi, inter, hybrid)