[Feature] Add MySQL Loader (#920)
Co-authored-by: Deven Patel <deven298@yahoo.com> Co-authored-by: Deshraj Yadav <deshrajdry@gmail.com>
This commit is contained in:
48
docs/data-sources/mysql.mdx
Normal file
48
docs/data-sources/mysql.mdx
Normal file
@@ -0,0 +1,48 @@
|
||||
---
|
||||
title: '🐬 MySQL'
|
||||
---
|
||||
|
||||
1. Setup the MySQL loader by configuring the SQL db.
|
||||
```Python
|
||||
from embedchain.loaders.mysql import MySQLLoader
|
||||
|
||||
config = {
|
||||
"host": "host",
|
||||
"port": "port",
|
||||
"database": "database",
|
||||
"user": "username",
|
||||
"password": "password",
|
||||
}
|
||||
|
||||
mysql_loader = MySQLLoader(config=config)
|
||||
```
|
||||
|
||||
For more details on how to setup with valid config, check MySQL [documentation](https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html).
|
||||
|
||||
2. Once you setup the loader, you can create an app and load data using the above MySQL loader
|
||||
```Python
|
||||
import os
|
||||
from embedchain.pipeline import Pipeline as App
|
||||
|
||||
app = App()
|
||||
|
||||
app.add("SELECT * FROM table_name;", data_type='mysql', loader=mysql_loader)
|
||||
# Adds `(1, 'What is your net worth, Elon Musk?', "As of October 2023, Elon Musk's net worth is $255.2 billion.")`
|
||||
|
||||
response = app.query(question)
|
||||
# Answer: As of October 2023, Elon Musk's net worth is $255.2 billion.
|
||||
```
|
||||
|
||||
NOTE: The `add` function of the app will accept any executable query to load data. DO NOT pass the `CREATE`, `INSERT` queries in `add` function.
|
||||
|
||||
3. We automatically create a chunker to chunk your SQL data, however if you wish to provide your own chunker class. Here is how you can do that:
|
||||
``Python
|
||||
|
||||
from embedchain.chunkers.mysql import MySQLChunker
|
||||
from embedchain.config.add_config import ChunkerConfig
|
||||
|
||||
mysql_chunker_config = ChunkerConfig(chunk_size=1000, chunk_overlap=0, length_function=len)
|
||||
mysql_chunker = MySQLChunker(config=mysql_chunker_config)
|
||||
|
||||
app.add("SELECT * FROM table_name;", data_type='mysql', loader=mysql_loader, chunker=mysql_chunker)
|
||||
```
|
||||
@@ -21,6 +21,7 @@ Embedchain comes with built-in support for various data sources. We handle the c
|
||||
<Card title="🎥📺 youtube video" href="/data-sources/youtube-video"></Card>
|
||||
<Card title="📬 Gmail" href="/data-sources/gmail"></Card>
|
||||
<Card title="🐘 Postgres" href="/data-sources/postgres"></Card>
|
||||
<Card title="🐬 MySQL" href="/data-sources/mysql"></Card>
|
||||
<Card title="🤖 Slack" href="/data-sources/slack"></Card>
|
||||
</CardGroup>
|
||||
|
||||
|
||||
Reference in New Issue
Block a user