Ravi Akasapu's GitHub Page
by ravi akasapu (raviakasapu@gmail.com)
With businesses adopting more and more complex and cutting edge technologies like IOT, Cloud for day to day operations and innovative methods to identify many business opportunities, there is a huge requirement to manage all the connected systems and maintain business data accurately and efficiently. As more and more systems are connected with the Operations, the complexity of managing the operational data is ever increasing. Adding to the technical complexity, there is also need to manage organizations overall strategy, business capabilities, competition within and outside Industry, Regulatory environment and other factors in managing overall data strategy.
Various types of data generated from Sensors and IOT devices, Time Series data from production and manufacturing systems, New Sales and Customer information, customer satisfaction surveys are few examples of Operational Data.
The data is being used in day to operational decision making for increased sales and revenue, cost reduction, increase efficiency, fraud detection, monitoring of the processes and to generate Business Intelligence.
The Operational data collected using various methods is further used in Analytics and ML of core data for different systems. This will in turn generate more business intelligence used as strategic decision making, creation of more products or improvement of various processes.
Since there are separate collection points of the data, organizations are Operational Data Stores and/or Business Warehouse to manage data. But the data collection points are separate this is still leading to data being stored in silos. For example, data collected by Sales team being used by sales and related team but may not be available to Inventory or Production. This leads to Multiple version of same data at the top. Even though there are cloud, database and RPA tools to manage the data variety, various BI tools making this difficult to consume data from other verticals. There are other challenges like cleaning, improving the data quality while delivering the data to BI Tools.
As most of the data is collected using different methods, sometimes using completely different business processes, it is very difficult to manage as a single version. Most of the time these data is analysed either as separate data or used without much integration with other similar systems. This leads to either duplication of data like same information is collected by different business processes or missing information in analysis by different business verticals.
There are additional challenges like Version control, Inconsistencies, Access Control, Security, redundancy, regulatory requirements which will affect the organizational data strategy. Each of these challenges need additional efforts, sometimes manual efforts to tackle and will lead to over or under utilization of resources and technology.
There are multiple use cases for AI/ML to use in operations. Using AI/ML will help in automating complex data tasks, automated data delivery to various Dashboards, reduce architecture complexity and help organizations moving towards Augmented Data Management.
Various AI/ML techniques are available in managing Database systems. This will improve organizing database queries, improve query accuracy and manage database infrastructure in a better way. This will improve the total operational efficiency and delivery of data to the end applications.
Other data cleaning and validation models like word embdeddings, entity detection, tracing, Labelled data and regression models can be applied on the text and unstructured data.
Using Deep Learning and other AI techniques can be applied on semi structured data for data cleaning, validation and also merging with other structured data. This will enhance the overall reporting and analytics.
Although there might be a need to include AI/ML in every business aspect, it has to be based on individual use case and to achieve defined outcomes. As more and more tools are developed in this space to address various issues there is also need to increase the awareness in utilizing AI/ML tools for Operational Data Management, especially when problems are not clearly defined. Using AI/ML without identifying the what is being achieved will result in negative results and re-work. That being said, there is more potential in addressing the issues on how organizations use data for day-to-day operations and AI/ML is best positioned to address those challenges of today and Future.
tags: