Overview

Pipeline actions

Pipeline information

Community

Blog

Neum AI

Components

Cloud

Dashboard

Support

Learn about how Neum AI lets you manage your pipelines.

Pipeline Management

Introduction

Quickstart

Connect Neum AI pipelines to your chatbot

Connect to chatbot

Deploy pipelines

Comparing local vs cloud capabilities for Neum AI

Local vs Cloud

Learn the high level architecture for Neum AI pipelines

Pipeline Architecture

Learn about how data is handled and trasnformed within the Neum AI pipeline

Data Flow

Learn about how data is extracted and pre-processed.

Data Pre-processing

Learn about how data is organized and used for search

Search

Evaluate the performance of your pipelines against a dataset.

Evaluation with dataset

Introduction to the Neum AI Cloud Platform

Neum AI Cloud

Support for data retrieval and feedback tracking

Data retrieval

Learn how the Neum AI platform provides logs and analytics to see the performance of your pipelines

Logs and Analytics

Leverage Langchain components with Neum AI

Langchain

Leverage LlamaIndex components with Neum AI

LlamaIndex

Pipeline

PipelineCollection

Connector to extract data from data sources

Source Connector

Website Connector

File Connector

Azure Blob Connector

S3 Connector

Postgres Connector

Sharepoint Connector

SingleStore Connector

Supabase Storage Connector

AutoLoader is a utility class in the Neum AI framework designed to automatically select and use the appropriate loader based on the type of file it encounters.

AutoLoader

HTMLLoader is designed to efficiently load and parse HTML files into NeumDocument objects for further processing within the Neum Ai ecosystem.

HTMLLoader

MarkdownLoader specializes in converting markdown files into NeumDocument objects, suitable for integration with Neumai’s document processing pipeline.

MarkdownLoader

The CSVLoader is dedicated to parsing CSV files and converting them into NeumDocument objects, with support for custom encoding and selective data extraction.

CSVLoader

JSONLoader is designed to parse JSON files and convert them into NeumDocument objects, catering to complex data structures and nested JSON.

JSONLoader

The PDFLoader class is designed to load and parse PDF files, transforming them into NeumDocument objects for further processing within the Neum AI platform.

PDFLoader

This class is responsible for chunking text data into smaller pieces based on character count, with optional overlapping between chunks.

CharacterChunker

The RecursiveChunker class is tailored for segmenting text into smaller chunks with a recursive approach, allowing for different levels of text granularity.

RecursiveChunker

CustomChunker is a flexible class designed to chunk text according to user-provided code, allowing for personalized text segmentation strategies.

CustomChunker

The OpenAIEmbed connector is designed to generate embeddings for text data using OpenAI’s models, providing a bridge between Neum AI documents and OpenAI’s powerful embedding capabilities.

OpenAIEmbed

The AzureOpenAIEmbed connector is designed to generate embeddings for text data using Azure OpenAI’s models, providing a bridge between Neum AI documents and OpenAI’s powerful embedding capabilities.

AzureOpenAIEmbed

ReplicateEmbed provides an interface to generate embeddings using models from the Replicate platform, integrated within the Neum AI framework for document processing.

ReplicateEmbed

The HuggingFaceEmbed connector is designed to generate embeddings for text data using Hugging Face hosted emedding models.

HuggingFaceEmbed

PineconeSink enables seamless integration with Pinecone’s vector database, supporting vector storage and similarity search for advanced data retrieval.

PineconeSink

QdrantSink integrates with Qdrant vector search engine to manage vector data, enabling powerful search capabilities and efficient data storage.

QdrantSink

SingleStoreSink enables the storage, retrieval, and semantic search of vectorized data in SingleStore databases, suitable for high-performance data operations.

SingleStoreSink

SupabaseSink is designed to store and search vectorized data within Supabase databases, enabling efficient similarity searches and data management.

SupabaseSink

WeaviateSink facilitates the storage and retrieval of vectorized data in the Weaviate vector database, allowing for efficient semantic search capabilities.

WeaviateSink

LanceDBSink enables seamless integration with LanceDB vector database, supporting vector storage and similarity search for advanced data retrieval.

LanceDBSink

MarqoSink enables seamless integration with Marqo vector database, supporting vector storage and similarity search for advanced data retrieval.

Actions	Description
Create	Pipeline record is created and tracked. First-run is executed.
Trigger	Run an existing pipeline. Can choose from full or delta sync modes.
Query	Query the data extracted in a given pipeline.
Query feedback	Record feedback for a pipeline query result.
Cancel	Cancel a pipeline run.

Get Started

Local development

Cloud platform

Integrations

Pipeline Management

Overview

Pipeline actions

Pipeline information

Get Started

Local development

Cloud platform

Integrations

​Overview

​Pipeline actions

​Pipeline information

Overview

Pipeline actions

Pipeline information