TechDogs-"Algolia Launches Next-Generation Crawler: Simplifying Data Ingestion For Developers"

Data Management

Algolia Launches Next-Generation Crawler: Simplifying Data Ingestion For Developers

By Business Wire

Business Wire
Overall Rating

SAN FRANCISCO--(BUSINESS WIRE)--Algolia, the world’s only end-to-end AI Search platform, today announced the availability of its next-generation Crawler, a critical tool rebuilt to enable developers to ingest data into Algolia AI Search more quickly and easily while spending less time on configuration and maintenance. This automated web scraping program simplifies and accelerates the ingestion of data by scanning web pages of any size.

A Powerful Tool for Seamless Data Ingestion

The next-generation Crawler is a powerful tool in data ingestion that effortlessly extracts content from an owned website. By navigating through linked pages, the Crawler quickly populates and updates Algolia AI Search indices when it detects changes automatically. The Crawler is tailored for developers seeking a fast and convenient method to implement search capabilities, freeing them up to work on their core product instead.

Tim Benniks, Developer Relations Leader at Outreach and Hygraph, said: “It’s really awesome that Algolia’s new Crawler parses schema.org formats on my website and recognizes that the content is actually a blog post. It just works!”

Streamlining Data Integration Across Multiple Sources

In scenarios where customers’ websites are managed by different teams or powered by various systems, the new Crawler shines as a highly convenient solution. It eliminates the traditionally time-consuming and resource-intensive processes associated with data integration, providing a swift time to value.

The Crawler can simplify data ingestion by sidestepping the complexities of building and maintaining an indexing pipeline. It is ideal in situations where the data structure is complex and there are limitations in the ability to send data to Algolia via API clients.

Philip Schmitt, Software Consultant, TNG Technology Consulting noted that the Self-Service Crawler will help developers extract, format, and upload content from multiple sites to the Algolia AI Search platform: “The Algolia Crawler is a great fit for our corporate website. It aggregates distributed content rapidly, and automatically and periodically updates aggregated content without adding any complexity to our codebase. Moreover, it enables accurate and efficient search through records, thus enhancing the search experience for users.”

Additionally, Algolia provides data connectors so that users can connect to popular sources such as Google Analytics, Adobe Analytics, and CSV formats to further enrich their records and improve the relevance in their search results.

Introducing Usage-Based Pricing

The reimagined Algolia Crawler will be available to all current Algolia Price plans. This will include 10,000 free crawls per month as part of Algolia’s ‘start for free’ Build program for developers. Developers can start experimenting with the new Crawler immediately and at no cost.

Bharat Guruprakash, Chief Product Officer, said: “The Algolia next-generation Crawler is available to developers, business users, and enterprise customers, and it will be integrated into the AI Search onboarding process. This easy-to-use and fast, low code solution enables data ingestion in minutes and is particularly beneficial for users who wish to build a proof of concept and experiment with it quickly, without initially uploading sensitive data.”

Getting started in minutes with No/Low code Configuration

The new Crawler features guided onboarding with intuitive step-by-step directions to facilitate a fast and easy setup. This streamlined approach includes a visual alternative to a traditional code editor – a No-Code Configurator for common settings (e.g., scheduling specific times for automatic crawls, listing of pages to be ignored, inputting login credentials for password-protected sites, and more).

Algolia continues to innovate and provides tools, such as the Self-Service Crawler, to streamline complex processes, enabling developers to focus on their core project. Developers can try the Self-Service Crawler and AI Search for free here.

About Algolia

Algolia is a leading provider of AI search solutions, serving over 17,000 businesses and 500,000 developers globally. Renowned for its user-friendly API-First platform and the fastest AI search technology, Algolia is the largest hosted search engine, trusted by businesses and developers for 1.75 trillion searches per year. Backed by a decade of innovation, expertise, and growth, Algolia continuously redefines the search landscape with its commitment to user-friendly solutions, significant scalability, and unmatched speed. Learn more: www.algolia.com.


Contacts

Media
Audrey Surette
PAN Communications for Algolia
algolia@pancomm.com

First published on Thu, Oct 3, 2024

Enjoyed what you've read so far? Great news - there's more to explore!

Stay up to date with the latest news, a vast collection of tech articles including introductory guides, product reviews, trends and more, thought-provoking interviews, hottest AI blogs and entertaining tech memes.

Plus, get access to branded insights such as informative white papers, intriguing case studies, in-depth reports, enlightening videos and exciting events and webinars from industry-leading global brands.

Dive into TechDogs' treasure trove today and Know Your World of technology!

Disclaimer - Reference to any specific product, software or entity does not constitute an endorsement or recommendation by TechDogs nor should any data or content published be relied upon. The views expressed by TechDogs' members and guests are their own and their appearance on our site does not imply an endorsement of them or any entity they represent. Views and opinions expressed by TechDogs' Authors are those of the Authors and do not necessarily reflect the view of TechDogs or any of its officials. All information / content found on TechDogs' site may not necessarily be reviewed by individuals with the expertise to validate its completeness, accuracy and reliability.

Join The Discussion

- Promoted By TechDogs -

Building AI-First Businesses: A CIO’s Guide To AI Adoption

Join Our Newsletter

Get weekly news, engaging articles, and career tips-all free!

By subscribing to our newsletter, you're cool with our terms and conditions and agree to our Privacy Policy.

  • Dark
  • Light