TechDogs-"Can The New Way Of Handling Big Data Solve Previous Challenges?"

Data Management

Can The New Way Of Handling Big Data Solve Previous Challenges?

By TD NewsDesk

TD NewsDesk

Updated on Wed, Apr 19, 2023

Overall Rating
Think about an old warehouse, a lake and a lakehouse! No, this isn’t the setting for a romantic novel or a horror-thriller. In fact, we’re talking about three kinds of management systems for big data!

So, let’s begin with the basics.

Currently, organizations use data warehouses and data lakes to store and maintain big data. Yet, they both possess certain drawbacks, which limit modern teams.

While data warehouses deal with structured data, they can be inflexible and costly. Similarly, data lakes offer a wider repository to store raw data which brings more flexibility to artificial intelligence and machine learning applications but they can be messy and tough to manage.

Up steps the new approach AKA data lakehouse, which consists of a single platform offering the best of both.

Adam Ronthal, a Data and Analytics Analyst at Gartner, believes the lakehouse approach has two primary aims, “One, to provide the right level of data optimization required to serve its target audience, and two, to physically converge the data warehouse and the data lake environment.”

TechDogs-"Image Of Adam Ronthal, Data And Analytics Analyst At Gartner"
However, while a data lakehouse might overcome challenges faced by data warehouses and data lakes, it also comes with its own set of drawbacks.

Some of these include delays, disruptions and higher costs, particularly where organizations already have existing architecture. Furthermore, migrating data can be time-consuming and costly. Additionally, issues such as compatibility and security may play a part.

According to Adrian Estala, VP, Field Chief Data Officer at Starburst, “If you have already moved a considerable amount of data into a data warehouse, you should develop a phased migration approach. This should minimize business disruption and prioritize data assets based on your analytics use cases.”

Estela believes that companies should first create a virtualization layer across existing environments and build virtual data products based on the current legacy warehouse structure. Then, companies should approach a carefully crafted migration plan based on costs, usage and requirements to ensure a smooth transition.

TechDogs-"Image Of Adrian Estala, VP, Field Chief Data Officer At Starburst"
Estela added, “On the front end, the data consumers shouldn’t care, and if you’re really good, some of them should not even be aware that the data was moved. The back end should be completely abstracted. What they should notice is easier access to reusable data products and much greater agility for iterating through improvements to their data solutions.”

Essentially, a data lakehouse may solve a range of problems but it must be a carefully considered move.

What do you think of the data lakehouse approach? Let us know in the comments below!

First published on Wed, Apr 19, 2023

Liked what you read? That’s only the tip of the tech iceberg!

Explore our vast collection of tech articles including introductory guides, product reviews, trends and more, stay up to date with the latest news, relish thought-provoking interviews and the hottest AI blogs, and tickle your funny bone with hilarious tech memes!

Plus, get access to branded insights from industry-leading global brands through informative white papers, engaging case studies, in-depth reports, enlightening videos and exciting events and webinars.

Dive into TechDogs' treasure trove today and Know Your World of technology like never before!

Disclaimer - Reference to any specific product, software or entity does not constitute an endorsement or recommendation by TechDogs nor should any data or content published be relied upon. The views expressed by TechDogs’ members and guests are their own and their appearance on our site does not imply an endorsement of them or any entity they represent. Views and opinions expressed by TechDogs’ Authors are those of the Authors and do not necessarily reflect the view of TechDogs or any of its officials. All information / content found on TechDogs’ site may not necessarily be reviewed by individuals with the expertise to validate its completeness, accuracy and reliability.

Tags:

Data Lakehouse Data Storage Data Warehouse Data Lakes

Join The Discussion

- Promoted By TechDogs -

The Brivo Partner Program
  • Dark
  • Light