Can AI Make Data Management Easier?

Businesses love data! (That’s an understatement) Their beloved data could pertain to customers, internal functions or even competitors! Although, raw data doesn’t do much good for them, which is why the data must undergo processing and refining before it becomes useful.

A while ago, when the data they collected wasn’t at the volumes it is today, data management was a much easier task and could be handled manually by a few experts. Since the introduction of cloud computing and enhanced data collection methods, data management has become a vital process for the effective and efficient functioning of businesses. This has seen the introduction of dedicated tools and artificial intelligence (AI) to the process.

According to Adam Ronthal, Research VP in Gartner’s ITL Data and Analytics Group, “Data management is no longer wholly focused on relational data. Document, graph, time-series, wide-column, key-value, ledger and other targeted data stores all provide specific optimizations for different types of data, and different use cases. Sometimes, these are combined in a single data management platform — a multimodel database; sometimes, they remain as best-fit, targeted point solutions.”

He believes that using AI to enhance data management would result in faster processes and cost-effectiveness for organizations. For example, in January 2023, Google and Aible (a company invested in imbibing AI into the data lifecycle) analyzed more than 75 datasets with more than 100 million rows of data and 150 million variable combinations. The cost? $80. One thousand times cheaper than traditional methods!


However, despite its capability to wrangle data, AI tools used in data lifecycle processes still need major tweaking, for which many in the industry believe we’re just scratching the surface.

While data scientists make it possible to build secure, non-biased and compliant models, not all organizations possess such experts. This is where they can make use of second-generation tools.

According to Jonathon Reilly, COO and Co-founder of Akkio – an AI company, “You no longer need to be a data engineer or data scientist to do complex data transformations — you can generate them with a large language model (LLM).”

At the same time, Noel Yuhanna, VP and Principal Analyst at Forrester Research, highlights an issue with using multiple tools, saying, “Every organization on the planet has more than two dozen data management tools. None of those tools talk to each other.”
   
Alternatively, organizations could make use of CDOs (Chief Data Officers) and data scientists to get a better understanding of their data infrastructure and source points, to utilize effective AI solutions. This reduces the need for human labor in routine data tasks.

In order to build trust with AI tools, Adam Ronthal suggests keeping human intervention with a gradually increasing reliance on AI tools.

“Start by [AI] making recommendations that are reviewed by humans. If those are correct and have the desired impact, eventually we will build trust and reduce the required level of supervision. Ultimately, we will reach a point where the AI has been right so many times, that we can allow it the autonomy to automate optimizations with minimal supervision. The stages of maturity can broadly be described as observing, reporting, recommending, optimizing and predicting. The last three are where augmentation is applied,” he said.

While AI is finding its way into data lifecycle management, it seems it’s a bit away from becoming a completely automated process and still needs human intervention to ensure effective delivery.
What do you think of AI in the data lifecycle process and where do you think such tools can be helpful? Let us know in the comments below!