Learn about how the Neum AI platform handles pipeline scheduling.

Scheduling

Neum AI

Introduction

Quickstart

Connect Neum AI pipelines to your chatbot

Connect to chatbot

Deploy pipelines

Comparing local vs cloud capabilities for Neum AI

Local vs Cloud

Learn the high level architecture for Neum AI pipelines

Pipeline Architecture

Learn about how data is handled and trasnformed within the Neum AI pipeline

Data Flow

Learn about how data is extracted and pre-processed.

Data Pre-processing

Learn about how data is organized and used for search

Search

Evaluate the performance of your pipelines against a dataset.

Evaluation with dataset

Introduction to the Neum AI Cloud Platform

Neum AI Cloud

Learn about how Neum AI lets you manage your pipelines.

Pipeline Management

Support for data retrieval and feedback tracking

Data retrieval

Learn how the Neum AI platform provides logs and analytics to see the performance of your pipelines

Logs and Analytics

Leverage Langchain components with Neum AI

Langchain

Leverage LlamaIndex components with Neum AI

LlamaIndex

Pipeline

PipelineCollection

Connector to extract data from data sources

Source Connector

The OpenAIEmbed connector is designed to generate embeddings for text data using OpenAI’s models, providing a bridge between Neum AI documents and OpenAI’s powerful embedding capabilities.

OpenAIEmbed

The AzureOpenAIEmbed connector is designed to generate embeddings for text data using Azure OpenAI’s models, providing a bridge between Neum AI documents and OpenAI’s powerful embedding capabilities.

AzureOpenAIEmbed

ReplicateEmbed provides an interface to generate embeddings using models from the Replicate platform, integrated within the Neum AI framework for document processing.

ReplicateEmbed

The HuggingFaceEmbed connector is designed to generate embeddings for text data using Hugging Face hosted emedding models.

HuggingFaceEmbed

PineconeSink enables seamless integration with Pinecone’s vector database, supporting vector storage and similarity search for advanced data retrieval.

PineconeSink

QdrantSink integrates with Qdrant vector search engine to manage vector data, enabling powerful search capabilities and efficient data storage.

QdrantSink

SingleStoreSink enables the storage, retrieval, and semantic search of vectorized data in SingleStore databases, suitable for high-performance data operations.

SingleStoreSink

SupabaseSink is designed to store and search vectorized data within Supabase databases, enabling efficient similarity searches and data management.

SupabaseSink

WeaviateSink facilitates the storage and retrieval of vectorized data in the Weaviate vector database, allowing for efficient semantic search capabilities.

WeaviateSink

LanceDBSink enables seamless integration with LanceDB vector database, supporting vector storage and similarity search for advanced data retrieval.

LanceDBSink

MarqoSink enables seamless integration with Marqo vector database, supporting vector storage and similarity search for advanced data retrieval.

MarqoSink

Selector

Neum Document

Neum Vector

Neum Search Result

Neum Client

Create a pipeline

Trigger a pipeline

Query the vector database attached to a pipeline

Query pipeline

Get pipeline configuration

Get the information for all pipelines for a user

Get pipelines for a user

Delete a pipeline

Get the information on the data stored in the sink of a pipeline

Get pipeline vector information

Get pipeline run

Get the information for a all pipeline runs for a given pipeline

Get pipelines runs

Cancel pipeline run

Get pipeline retrievals

For a given retrieval provide feedback on it.

Provide retrieval feedback

Community

Blog

Components

Cloud

Dashboard

Support

Website Connector

File Connector

Azure Blob Connector

S3 Connector

Postgres Connector

Sharepoint Connector

SingleStore Connector

Supabase Storage Connector

AutoLoader is a utility class in the Neum AI framework designed to automatically select and use the appropriate loader based on the type of file it encounters.

AutoLoader

HTMLLoader is designed to efficiently load and parse HTML files into NeumDocument objects for further processing within the Neum Ai ecosystem.

HTMLLoader

MarkdownLoader specializes in converting markdown files into NeumDocument objects, suitable for integration with Neumai’s document processing pipeline.

MarkdownLoader

The CSVLoader is dedicated to parsing CSV files and converting them into NeumDocument objects, with support for custom encoding and selective data extraction.

CSVLoader

JSONLoader is designed to parse JSON files and convert them into NeumDocument objects, catering to complex data structures and nested JSON.

JSONLoader

The PDFLoader class is designed to load and parse PDF files, transforming them into NeumDocument objects for further processing within the Neum AI platform.

PDFLoader

This class is responsible for chunking text data into smaller pieces based on character count, with optional overlapping between chunks.

CharacterChunker

The RecursiveChunker class is tailored for segmenting text into smaller chunks with a recursive approach, allowing for different levels of text granularity.

RecursiveChunker

CustomChunker is a flexible class designed to chunk text according to user-provided code, allowing for personalized text segmentation strategies.

Get Started

Local development

Cloud platform

Integrations

Scheduling