klas/t6_mem0
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rename embedchain to mem0 and open sourcing code for long term memory (#1474)

Browse Source 
Co-authored-by: Deshraj Yadav <deshrajdry@gmail.com>
This commit is contained in:
Taranjeet Singh
2024-07-12 07:51:33 -07:00
committed by GitHub
parent 83e8c97295
commit f842a92e25
665 changed files with 9427 additions and 6592 deletions
Download Patch File Download Diff File Expand all files Collapse all files

161
embedchain/notebooks/anthropic.ipynb Normal file
View File

@@ -0,0 +1,161 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using Anthropic with Embedchain\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-NbXjAdlh0vJ",
"outputId": "efdce0dc-fb30-4e01-f5a8-ef1a7f4e8c09"
},
"outputs": [],
"source": [
"!pip install embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set Anthropic related environment variables\n",
"\n",
"You can find `OPENAI_API_KEY` on your [OpenAI dashboard](https://platform.openai.com/account/api-keys) and `ANTHROPIC_API_KEY` on your [Anthropic dashboard](https://console.anthropic.com/account/keys)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\"\n",
"os.environ[\"ANTHROPIC_API_KEY\"] = \"xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3: Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"provider\": \"anthropic\",\n",
" \"config\": {\n",
" \"model\": \"claude-instant-1\",\n",
" \"temperature\": 0.5,\n",
" \"top_p\": 1,\n",
" \"stream\": False\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"id": "Sn_0rx9QjIY9",
"outputId": "dc17baec-39b5-4dc8-bd42-f2aad92697eb"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 391
},
"id": "cvIK7dWRjN_f",
"outputId": "3d1cb7ce-969e-4dad-d48c-b818b7447cc0"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

174
embedchain/notebooks/azure-openai.ipynb Normal file
View File

@@ -0,0 +1,174 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "63ab5e89",
"metadata": {},
"source": [
"## Cookbook for using Azure OpenAI with Embedchain"
]
},
{
"cell_type": "markdown",
"id": "e32a0265",
"metadata": {},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b80ff15a",
"metadata": {},
"outputs": [],
"source": [
"!pip install embedchain"
]
},
{
"cell_type": "markdown",
"id": "ac982a56",
"metadata": {},
"source": [
"### Step-2: Set Azure OpenAI related environment variables\n",
"\n",
"You can find these env variables on your Azure OpenAI dashboard."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "e0a36133",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_TYPE\"] = \"azure\"\n",
"os.environ[\"OPENAI_API_BASE\"] = \"https://xxx.openai.azure.com/\"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"xxx\"\n",
"os.environ[\"OPENAI_API_VERSION\"] = \"xxx\""
]
},
{
"cell_type": "markdown",
"id": "7d7b554e",
"metadata": {},
"source": [
"### Step-3: Define your llm and embedding model config"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "b9f52fc5",
"metadata": {},
"outputs": [],
"source": [
"config = \"\"\"\n",
"llm:\n",
" provider: azure_openai\n",
" model: gpt-35-turbo\n",
" config:\n",
" deployment_name: ec_openai_azure\n",
" temperature: 0.5\n",
" max_tokens: 1000\n",
" top_p: 1\n",
" stream: false\n",
"\n",
"embedder:\n",
" provider: azure_openai\n",
" config:\n",
" model: text-embedding-ada-002\n",
" deployment_name: ec_embeddings_ada_002\n",
"\"\"\"\n",
"\n",
"# Write the multi-line string to a YAML file\n",
"with open('azure_openai.yaml', 'w') as file:\n",
" file.write(config)"
]
},
{
"cell_type": "markdown",
"id": "98a11130",
"metadata": {},
"source": [
"### Step-4 Create embedchain app based on the config"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1ee9bdd9",
"metadata": {},
"outputs": [],
"source": [
"app = App.from_config(config_path=\"azure_openai.yaml\")"
]
},
{
"cell_type": "markdown",
"id": "554dc97b",
"metadata": {},
"source": [
"### Step-5: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "686ae765",
"metadata": {},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"id": "ccc7d421",
"metadata": {},
"source": [
"### Step-6: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "27868a7d",
"metadata": {},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

16
embedchain/notebooks/azure_openai.yaml Normal file
View File

@@ -0,0 +1,16 @@
llm:
provider: azure_openai
model: gpt-35-turbo
config:
deployment_name: ec_openai_azure
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
embedder:
provider: azure_openai
config:
model: text-embedding-ada-002
deployment_name: ec_embeddings_ada_002

147
embedchain/notebooks/chromadb.ipynb Normal file
View File

@@ -0,0 +1,147 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using ChromaDB with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-NbXjAdlh0vJ"
},
"outputs": [],
"source": [
"!pip install embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set OpenAI environment variables\n",
"\n",
"You can find this env variable on your [OpenAI dashboard](https://platform.openai.com/account/api-keys)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"vectordb\": {\n",
" \"provider\": \"chroma\",\n",
" \"config\": {\n",
" \"collection_name\": \"my-collection\",\n",
" \"host\": \"your-chromadb-url.com\",\n",
" \"port\": 5200,\n",
" \"allow_reset\": True\n",
" }\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Sn_0rx9QjIY9"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cvIK7dWRjN_f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

135
embedchain/notebooks/clarifai.ipynb Normal file
View File

@@ -0,0 +1,135 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Cookbook for using Clarifai LLM and Embedders with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step-1: Install embedchain-clarifai package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install embedchain[clarifai]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step-2: Set Clarifai PAT as env variable.\n",
"Sign-up to [Clarifai](https://clarifai.com/signup?utm_source=clarifai_home&utm_medium=direct&) platform and you can obtain `CLARIFAI_PAT` by following this [link](https://docs.clarifai.com/clarifai-basics/authentication/personal-access-tokens/).\n",
"\n",
"optionally you can also pass `api_key` in config of llm/embedder class."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"CLARIFAI_PAT\"]=\"xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step-3 Create embedchain app using clarifai LLM and embedder and define your config.\n",
"\n",
"Browse through Clarifai community page to get the URL of different [LLM](https://clarifai.com/explore/models?page=1&perPage=24&filterData=%5B%7B%22field%22%3A%22use_cases%22%2C%22value%22%3A%5B%22llm%22%5D%7D%5D) and [embedding](https://clarifai.com/explore/models?page=1&perPage=24&filterData=%5B%7B%22field%22%3A%22input_fields%22%2C%22value%22%3A%5B%22text%22%5D%7D%2C%7B%22field%22%3A%22output_fields%22%2C%22value%22%3A%5B%22embeddings%22%5D%7D%5D) models available."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Use model_kwargs to pass all model specific parameters for inference.\n",
"app = App.from_config(config={\n",
" \"llm\": {\n",
" \"provider\": \"clarifai\",\n",
" \"config\": {\n",
" \"model\": \"https://clarifai.com/mistralai/completion/models/mistral-7B-Instruct\",\n",
" \"model_kwargs\": {\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000\n",
" }\n",
" }\n",
" },\n",
" \"embedder\": {\n",
" \"provider\": \"clarifai\",\n",
" \"config\": {\n",
" \"model\": \"https://clarifai.com/openai/embed/models/text-embedding-ada\",\n",
" }\n",
"}\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "v1",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.9.10"
}
},
"nbformat": 4,
"nbformat_minor": 2
}

165
embedchain/notebooks/cohere.ipynb Normal file
View File

@@ -0,0 +1,165 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using Cohere with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-NbXjAdlh0vJ",
"outputId": "fae77912-4e6a-4c78-fcb7-fbbe46f7a9c7"
},
"outputs": [],
"source": [
"!pip install embedchain[cohere]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set Cohere related environment variables\n",
"\n",
"You can find `OPENAI_API_KEY` on your [OpenAI dashboard](https://platform.openai.com/account/api-keys) and `COHERE_API_KEY` key on your [Cohere dashboard](https://dashboard.cohere.com/api-keys)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\"\n",
"os.environ[\"COHERE_API_KEY\"] = \"xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 321
},
"id": "Amzxk3m-i3tD",
"outputId": "afe8afde-5cb8-46bc-c541-3ad26cc3fa6e"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"provider\": \"cohere\",\n",
" \"config\": {\n",
" \"model\": \"gptd-instruct-tft\",\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000,\n",
" \"top_p\": 1,\n",
" \"stream\": False\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 176
},
"id": "Sn_0rx9QjIY9",
"outputId": "2f2718a4-3b7e-4844-fd46-3e0857653ca0"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cvIK7dWRjN_f",
"outputId": "79e873c8-9594-45da-f5a3-0a893511267f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

145
embedchain/notebooks/elasticsearch.ipynb Normal file
View File

@@ -0,0 +1,145 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using ElasticSearchDB with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-NbXjAdlh0vJ"
},
"outputs": [],
"source": [
"!pip install embedchain[elasticsearch]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set OpenAI environment variables.\n",
"\n",
"You can find this env variable on your [OpenAI dashboard](https://platform.openai.com/account/api-keys)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"provider\": \"elasticsearch\",\n",
" \"config\": {\n",
" \"collection_name\": \"es-index\",\n",
" \"es_url\": \"your-elasticsearch-url.com\",\n",
" \"allow_reset\": True,\n",
" \"api_key\": \"xxx\"\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Sn_0rx9QjIY9"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cvIK7dWRjN_f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

111
embedchain/notebooks/embedchain-chromadb-server.ipynb Normal file
View File

@@ -0,0 +1,111 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "553f2e71",
"metadata": {},
"source": [
"## Embedchain chromadb server example"
]
},
{
"cell_type": "markdown",
"id": "513e12e6",
"metadata": {},
"source": [
"This notebook shows an example of how you can use embedchain with chromdb (server). \n",
"\n",
"\n",
"First, run chroma inside docker using the following command:\n",
"\n",
"\n",
"```bash\n",
"git clone https://github.com/chroma-core/chroma\n",
"cd chroma && docker-compose up -d --build\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "92e7ad71",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"from embedchain.config import AppConfig\n",
"\n",
"\n",
"chromadb_host = \"localhost\"\n",
"chromadb_port = 8000\n",
"\n",
"config = AppConfig(host=chromadb_host, port=chromadb_port)\n",
"elon_bot = App(config)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "1a6d6841",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"All data from https://en.wikipedia.org/wiki/Elon_Musk already exists in the database.\n",
"All data from https://www.tesla.com/elon-musk already exists in the database.\n"
]
}
],
"source": [
"# Embed Online Resources\n",
"elon_bot.add(\"web_page\", \"https://en.wikipedia.org/wiki/Elon_Musk\")\n",
"elon_bot.add(\"web_page\", \"https://www.tesla.com/elon-musk\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "34cda99c",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Elon Musk runs four companies: Tesla, SpaceX, Neuralink, and The Boring Company.'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"elon_bot.query(\"How many companies does Elon Musk run?\")"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.8"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

121
embedchain/notebooks/embedchain-docs-site-example.ipynb Normal file
View File

@@ -0,0 +1,121 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "e9a9dc6a",
"metadata": {},
"outputs": [],
"source": [
"from embedchain import App\n",
"\n",
"embedchain_docs_bot = App()"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "c1c24d68",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"All data from https://docs.embedchain.ai/ already exists in the database.\n"
]
}
],
"source": [
"embedchain_docs_bot.add(\"docs_site\", \"https://docs.embedchain.ai/\")"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "48cdaecf",
"metadata": {},
"outputs": [],
"source": [
"answer = embedchain_docs_bot.query(\"Write a flask API for embedchain bot\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"id": "0fe18085",
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"To write a Flask API for the embedchain bot, you can use the following code snippet:\n",
"\n",
"```python\n",
"from flask import Flask, request, jsonify\n",
"from embedchain import App\n",
"\n",
"app = Flask(__name__)\n",
"bot = App()\n",
"\n",
"# Add datasets to the bot\n",
"bot.add(\"youtube_video\", \"https://www.youtube.com/watch?v=3qHkcs3kG44\")\n",
"bot.add(\"pdf_file\", \"https://navalmanack.s3.amazonaws.com/Eric-Jorgenson_The-Almanack-of-Naval-Ravikant_Final.pdf\")\n",
"\n",
"@app.route('/query', methods=['POST'])\n",
"def query():\n",
" data = request.get_json()\n",
" question = data['question']\n",
" response = bot.query(question)\n",
" return jsonify({'response': response})\n",
"\n",
"if __name__ == '__main__':\n",
" app.run()\n",
"```\n",
"\n",
"In this code, we create a Flask app and initialize an instance of the embedchain bot. We then add the desired datasets to the bot using the `add()` function.\n",
"\n",
"Next, we define a route `/query` that accepts POST requests. The request body should contain a JSON object with a `question` field. The bot's `query()` function is called with the provided question, and the response is returned as a JSON object.\n",
"\n",
"Finally, we run the Flask app using `app.run()`.\n",
"\n",
"Note: Make sure to install Flask and embedchain packages before running this code."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from IPython.display import Markdown\n",
"# Create a Markdown object and display it\n",
"markdown_answer = Markdown(answer)\n",
"display(markdown_answer)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

169
embedchain/notebooks/gpt4all.ipynb Normal file
View File

@@ -0,0 +1,169 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using GPT4All with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-NbXjAdlh0vJ",
"outputId": "077fa470-b51f-4c29-8c22-9c5f0a9cef47"
},
"outputs": [],
"source": [
"!pip install embedchain[opensource]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set GPT4ALL related environment variables\n",
"\n",
"GPT4All is free for all and doesn't require any API Key to use it. So you can use it for free!"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"from embedchain import App"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Amzxk3m-i3tD",
"outputId": "775db99b-e217-47db-f87f-788495d86f26"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"llm\": {\n",
" \"provider\": \"gpt4all\",\n",
" \"config\": {\n",
" \"model\": \"orca-mini-3b-gguf2-q4_0.gguf\",\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000,\n",
" \"top_p\": 1,\n",
" \"stream\": False\n",
" }\n",
" },\n",
" \"embedder\": {\n",
" \"provider\": \"gpt4all\",\n",
" \"config\": {\n",
" \"model\": \"all-MiniLM-L6-v2\"\n",
" }\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"id": "Sn_0rx9QjIY9",
"outputId": "c6514f17-3cb2-4fbc-c80d-79b3a311ff30"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 480
},
"id": "cvIK7dWRjN_f",
"outputId": "c74f356a-d2fb-426d-b36c-d84911397338"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

168
embedchain/notebooks/hugging_face_hub.ipynb Normal file
View File

@@ -0,0 +1,168 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using Hugging Face Hub with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "-NbXjAdlh0vJ",
"outputId": "35ddc904-8067-44cf-dcc9-3c8b4cd29989"
},
"outputs": [],
"source": [
"!pip install embedchain[huggingface_hub,opensource]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set Hugging Face Hub related environment variables\n",
"\n",
"You can find your `HUGGINGFACE_ACCESS_TOKEN` key on your [Hugging Face Hub dashboard](https://huggingface.co/settings/tokens)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"HUGGINGFACE_ACCESS_TOKEN\"] = \"hf_xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"llm\": {\n",
" \"provider\": \"huggingface\",\n",
" \"config\": {\n",
" \"model\": \"google/flan-t5-xxl\",\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000,\n",
" \"top_p\": 0.8,\n",
" \"stream\": False\n",
" }\n",
" },\n",
" \"embedder\": {\n",
" \"provider\": \"huggingface\",\n",
" \"config\": {\n",
" \"model\": \"sentence-transformers/all-mpnet-base-v2\"\n",
" }\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 70
},
"id": "Sn_0rx9QjIY9",
"outputId": "3c2a803a-3a93-4b0d-a6ae-17ae3c96c3c2"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cvIK7dWRjN_f",
"outputId": "47a89d1c-b322-495c-822a-6c2ecef894d2"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

165
embedchain/notebooks/jina.ipynb Normal file
View File

@@ -0,0 +1,165 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using JinaChat with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "-NbXjAdlh0vJ",
"outputId": "69cb79a6-c758-4656-ccf7-9f3105c81d16"
},
"outputs": [],
"source": [
"!pip install embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set JinaChat related environment variables\n",
"\n",
"You can find `OPENAI_API_KEY` on your [OpenAI dashboard](https://platform.openai.com/account/api-keys) and `JINACHAT_API_KEY` key on your [Chat Jina dashboard](https://chat.jina.ai/api)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\"\n",
"os.environ[\"JINACHAT_API_KEY\"] = \"xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 321
},
"id": "Amzxk3m-i3tD",
"outputId": "8d00da74-5f73-49bb-b868-dcf1c375ac85"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"provider\": \"jina\",\n",
" \"config\": {\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000,\n",
" \"top_p\": 1,\n",
" \"stream\": False\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"id": "Sn_0rx9QjIY9",
"outputId": "10eeacc7-9263-448e-876d-002af897ebe5"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cvIK7dWRjN_f",
"outputId": "7dc7212f-a0e9-43c8-f119-f595ba79b4b7"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

146
embedchain/notebooks/lancedb.ipynb Normal file
View File

@@ -0,0 +1,146 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using LanceDB with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-NbXjAdlh0vJ"
},
"outputs": [],
"source": [
"! pip install embedchain lancedb"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set environment variables needed for LanceDB\n",
"\n",
"You can find this env variable on your [OpenAI](https://platform.openai.com/account/api-keys)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"vectordb\": {\n",
" \"provider\": \"lancedb\",\n",
" \"config\": {\n",
" \"collection_name\": \"lancedb-index\"\n",
" }\n",
" }\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Sn_0rx9QjIY9"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cvIK7dWRjN_f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

161
embedchain/notebooks/llama2.ipynb Normal file
View File

@@ -0,0 +1,161 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using LLAMA2 with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-NbXjAdlh0vJ",
"outputId": "86a4a9b2-4ed6-431c-da6f-c3eacb390f42"
},
"outputs": [],
"source": [
"!pip install embedchain[llama2]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set LLAMA2 related environment variables\n",
"\n",
"You can find `OPENAI_API_KEY` on your [OpenAI dashboard](https://platform.openai.com/account/api-keys) and `REPLICATE_API_TOKEN` key on your [Replicate dashboard](https://replicate.com/account/api-tokens)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\"\n",
"os.environ[\"REPLICATE_API_TOKEN\"] = \"xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"provider\": \"llama2\",\n",
" \"config\": {\n",
" \"model\": \"a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5\",\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000,\n",
" \"top_p\": 0.5,\n",
" \"stream\": False\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"id": "Sn_0rx9QjIY9",
"outputId": "ba158e9c-0f16-4c6b-a876-7543120985a2"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 599
},
"id": "cvIK7dWRjN_f",
"outputId": "e2d11a25-a2ed-4034-ec6a-e8a5986c89ae"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

207
embedchain/notebooks/ollama.ipynb Normal file
View File

@@ -0,0 +1,207 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using Ollama with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Setup Ollama, follow these instructions https://github.com/jmorganca/ollama\n",
"\n",
"Once Setup is done:\n",
"\n",
"- ollama pull llama2 (All supported models can be found here: https://ollama.ai/library)\n",
"- ollama run llama2 (Test out the model once)\n",
"- ollama serve"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-2 Create embedchain app and define your config (all local inference)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 321
},
"id": "Amzxk3m-i3tD",
"outputId": "afe8afde-5cb8-46bc-c541-3ad26cc3fa6e"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"/Users/sukkritsharma/workspace/embedchain/.venv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
" from .autonotebook import tqdm as notebook_tqdm\n"
]
}
],
"source": [
"from embedchain import App\n",
"app = App.from_config(config={\n",
" \"llm\": {\n",
" \"provider\": \"ollama\",\n",
" \"config\": {\n",
" \"model\": \"llama2\",\n",
" \"temperature\": 0.5,\n",
" \"top_p\": 1,\n",
" \"stream\": True\n",
" }\n",
" },\n",
" \"embedder\": {\n",
" \"provider\": \"huggingface\",\n",
" \"config\": {\n",
" \"model\": \"BAAI/bge-small-en-v1.5\"\n",
" }\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-3: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 176
},
"id": "Sn_0rx9QjIY9",
"outputId": "2f2718a4-3b7e-4844-fd46-3e0857653ca0"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Inserting batches in chromadb: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1.57it/s]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Successfully saved https://www.forbes.com/profile/elon-musk (DataType.WEB_PAGE). New chunks count: 4\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n"
]
},
{
"data": {
"text/plain": [
"'8cf46026cabf9b05394a2658bd1fe890'"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-4: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cvIK7dWRjN_f",
"outputId": "79e873c8-9594-45da-f5a3-0a893511267f"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Elon Musk is a business magnate, investor, and engineer. He is the CEO of SpaceX and Tesla, Inc., and has been involved in other successful ventures such as Neuralink and The Boring Company. Musk is known for his innovative ideas, entrepreneurial spirit, and vision for the future of humanity.\n",
"\n",
"As the CEO of Tesla, Musk has played a significant role in popularizing electric vehicles and making them more accessible to the masses. Under his leadership, Tesla has grown into one of the most valuable companies in the world.\n",
"\n",
"SpaceX, another company founded by Musk, is a leading player in the commercial space industry. SpaceX has developed advanced rockets and spacecraft, including the Falcon 9 and Dragon, which have successfully launched numerous satellites and other payloads into orbit.\n",
"\n",
"Musk is also known for his ambitious goals, such as establishing a human settlement on Mars and developing sustainable energy solutions to address climate change. He has been recognized for his philanthropic efforts, particularly in the area of education, and has been awarded numerous honors and awards for his contributions to society.\n",
"\n",
"Overall, Elon Musk is a highly influential and innovative entrepreneur who has made significant impacts in various industries and has inspired many people around the world with his vision and leadership."
]
}
],
"source": [
"answer = app.query(\"who is elon musk?\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.9"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

160
embedchain/notebooks/openai.ipynb Normal file
View File

@@ -0,0 +1,160 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using OpenAI with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"id": "-NbXjAdlh0vJ",
"outputId": "6c630676-c7fc-4054-dc94-c613de58a037"
},
"outputs": [],
"source": [
"!pip install embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set OpenAI environment variables\n",
"\n",
"You can find this env variable on your [OpenAI dashboard](https://platform.openai.com/account/api-keys)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"llm\": {\n",
" \"provider\": \"openai\",\n",
" \"config\": {\n",
" \"model\": \"gpt-3.5-turbo\",\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000,\n",
" \"top_p\": 1,\n",
" \"stream\": False\n",
" }\n",
" },\n",
" \"embedder\": {\n",
" \"provider\": \"openai\",\n",
" \"config\": {\n",
" \"model\": \"text-embedding-ada-002\"\n",
" }\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Sn_0rx9QjIY9"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cvIK7dWRjN_f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.11.6"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

16
embedchain/notebooks/openai_azure.yaml Normal file
View File

@@ -0,0 +1,16 @@
llm:
provider: azure_openai
model: gpt-35-turbo
config:
deployment_name: ec_openai_azure
temperature: 0.5
max_tokens: 1000
top_p: 1
stream: false
embedder:
provider: azure_openai
config:
model: text-embedding-ada-002
deployment_name: ec_embeddings_ada_002

147
embedchain/notebooks/opensearch.ipynb Normal file
View File

@@ -0,0 +1,147 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using OpenSearchDB with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-NbXjAdlh0vJ"
},
"outputs": [],
"source": [
"!pip install embedchain[opensearch]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set OpenAI environment variables and install the dependencies.\n",
"\n",
"You can find this env variable on your [OpenAI dashboard](https://platform.openai.com/account/api-keys). Now lets install the dependencies needed for Opensearch."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"provider\": \"opensearch\",\n",
" \"config\": {\n",
" \"opensearch_url\": \"your-opensearch-url.com\",\n",
" \"http_auth\": [\"admin\", \"admin\"],\n",
" \"vector_dimension\": 1536,\n",
" \"collection_name\": \"my-app\",\n",
" \"use_ssl\": False,\n",
" \"verify_certs\": False\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Sn_0rx9QjIY9"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cvIK7dWRjN_f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

146
embedchain/notebooks/pinecone.ipynb Normal file
View File

@@ -0,0 +1,146 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using PineconeDB with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-NbXjAdlh0vJ"
},
"outputs": [],
"source": [
"!pip install embedchain pinecone-client pinecone-text"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set environment variables needed for Pinecone\n",
"\n",
"You can find this env variable on your [OpenAI dashboard](https://platform.openai.com/account/api-keys) and [Pinecone dashboard](https://app.pinecone.io/)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\"\n",
"os.environ[\"PINECONE_API_KEY\"] = \"xxx\"\n",
"os.environ[\"PINECONE_ENV\"] = \"xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Amzxk3m-i3tD"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"provider\": \"pinecone\",\n",
" \"config\": {\n",
" \"metric\": \"cosine\",\n",
" \"vector_dimension\": 768,\n",
" \"collection_name\": \"pc-index\"\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Sn_0rx9QjIY9"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cvIK7dWRjN_f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

211
embedchain/notebooks/together.ipynb Normal file
View File

@@ -0,0 +1,211 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using Cohere with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-NbXjAdlh0vJ",
"outputId": "fae77912-4e6a-4c78-fcb7-fbbe46f7a9c7"
},
"outputs": [],
"source": [
"!pip install embedchain[together]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set Cohere related environment variables\n",
"\n",
"You can find `OPENAI_API_KEY` on your [OpenAI dashboard](https://platform.openai.com/account/api-keys) and `TOGETHER_API_KEY` key on your [Together dashboard](https://api.together.xyz/settings/api-keys)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"\"\n",
"os.environ[\"TOGETHER_API_KEY\"] = \"\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 321
},
"id": "Amzxk3m-i3tD",
"outputId": "afe8afde-5cb8-46bc-c541-3ad26cc3fa6e"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"provider\": \"together\",\n",
" \"config\": {\n",
" \"model\": \"mistralai/Mixtral-8x7B-Instruct-v0.1\",\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 176
},
"id": "Sn_0rx9QjIY9",
"outputId": "2f2718a4-3b7e-4844-fd46-3e0857653ca0"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Inserting batches in chromadb: 100%|██████████| 1/1 [00:01<00:00, 1.16s/it]"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Successfully saved https://www.forbes.com/profile/elon-musk (DataType.WEB_PAGE). New chunks count: 4\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n"
]
},
{
"data": {
"text/plain": [
"'8cf46026cabf9b05394a2658bd1fe890'"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cvIK7dWRjN_f",
"outputId": "79e873c8-9594-45da-f5a3-0a893511267f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 0
}

162
embedchain/notebooks/vertex_ai.ipynb Normal file
View File

@@ -0,0 +1,162 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "b02n_zJ_hl3d"
},
"source": [
"## Cookbook for using VertexAI with Embedchain"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gyJ6ui2vhtMY"
},
"source": [
"### Step-1: Install embedchain package"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "-NbXjAdlh0vJ",
"outputId": "eb9be5b6-dc81-43d2-d515-df8f0116be11"
},
"outputs": [],
"source": [
"!pip install embedchain[vertexai]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nGnpSYAAh2bQ"
},
"source": [
"### Step-2: Set VertexAI related environment variables\n",
"\n",
"You can find `OPENAI_API_KEY` on your [OpenAI dashboard](https://platform.openai.com/account/api-keys)."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0fBdQ9GAiRvK"
},
"outputs": [],
"source": [
"import os\n",
"from embedchain import App\n",
"\n",
"os.environ[\"OPENAI_API_KEY\"] = \"sk-xxx\""
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGt6uPLIi1CS"
},
"source": [
"### Step-3 Create embedchain app and define your config"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 582
},
"id": "Amzxk3m-i3tD",
"outputId": "5084b6ea-ec20-4281-9f36-e21e93c17475"
},
"outputs": [],
"source": [
"app = App.from_config(config={\n",
" \"llm\": {\n",
" \"provider\": \"vertexai\",\n",
" \"config\": {\n",
" \"model\": \"chat-bison\",\n",
" \"temperature\": 0.5,\n",
" \"max_tokens\": 1000,\n",
" \"stream\": False\n",
" }\n",
" },\n",
" \"embedder\": {\n",
" \"provider\": \"vertexai\",\n",
" \"config\": {\n",
" \"model\": \"textembedding-gecko\"\n",
" }\n",
" }\n",
"})"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XNXv4yZwi7ef"
},
"source": [
"### Step-4: Add data sources to your app"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Sn_0rx9QjIY9"
},
"outputs": [],
"source": [
"app.add(\"https://www.forbes.com/profile/elon-musk\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7W6fDeAjMAP"
},
"source": [
"### Step-5: All set. Now start asking questions related to your data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cvIK7dWRjN_f"
},
"outputs": [],
"source": [
"while(True):\n",
" question = input(\"Enter question: \")\n",
" if question in ['q', 'exit', 'quit']:\n",
" break\n",
" answer = app.query(question)\n",
" print(answer)"
]
}
],
"metadata": {
"colab": {
"provenance": []
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 0
}