Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tutorial for using Asymmetric models #3258

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
+532 −0
Open

Tutorial for using Asymmetric models #3258

Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
f605636
adds initial tutorial contents
brianf-aws Dec 6, 2024
bcdd180
Add: initial thoughts to highlevel steps
brianf-aws Dec 6, 2024
8cd8359
expand more details on tutorial
brianf-aws Dec 6, 2024
c792403
fix typo
brianf-aws Dec 6, 2024
d48f75c
adds Asymmetric semantic search
brianf-aws Dec 17, 2024
f4ac9a0
Apply Code/Writing review
brianf-aws Dec 23, 2024
47bf8ca
Removed Docker requirement
brianf-aws Dec 26, 2024
b11cbe1
Make clear OpenSearch is ran locally in tutorial
brianf-aws Dec 27, 2024
c43e9a5
Simplify explanation of file location
brianf-aws Dec 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 11 additions & 20 deletions docs/tutorials/semantic_search/asymmetric_embedding_model.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,36 +1,27 @@
# Tutorial: Running Asymmetric Semnantic Search within OpenSearch

This tutorial demonstrates how generating text embeddings using an asymmetric embedding model in OpenSearch. The embeddings will be used
to run semantic search, implemented using a Docker container. The example model used in this tutorial is the multilingual
`intfloat/multilingual-e5-small` model from Hugging Face.
You will learn how to prepare the model, register it in OpenSearch, and run inference to generate embeddings.
This tutorial demonstrates how generating text embeddings using an asymmetric embedding model in OpenSearch. The example model used in this tutorial is the multilingual
`intfloat/multilingual-e5-small` model from Hugging Face.
In this tutorial, you'll learn how to prepare the model, register it in OpenSearch, and run inference to generate embeddings.

> **Note**: Make sure to replace all placeholders (e.g., `your_`) with your specific values.

---

## Prerequisites

- Docker Desktop installed and running on your local machine.
- Basic familiarity with Docker and OpenSearch.
- OpenSearch installed on your machine
- Access to the Hugging Face `intfloat/multilingual-e5-small` model (or another model of your choice).
- Basic knowledge of Linux commands
---

## Step 1: Spin up a Docker OpenSearch cluster
## Step 1: Start OpenSearch locally
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we care if customer started opensearch locally or with public IP? In our blue print example, we setup OS locally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally locally because I have a step were I make them use python to service the model which runs on localhost:8080

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if I have a OS which is running in let's say 8.8.8.8 and then if I use 8.8.8.8:8080, won't that work?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not an expert but if two processes run on the same address like localhost but use different ports then it should be fine. Also the python server is just a one time occurrence just a means to get OS to download the model.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should work. If we setup a OS cluster in EC2 host, we can access through the public URL and the port. So I would suggest to rephrase: Start Opensearch and in line 22 we can say:

Run OpenSearch and ensure the following steps are completed. In this example, we set up the OS cluster locally.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thank you for the suggestion (addressed in commit b11cbe1).


To run OpenSearch in a local development environment, you can use Docker and a preconfigured `docker-compose` file.
See here for directions to install and run [OpenSearch](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/).

### a. Create a Docker Compose File
Run OpenSearch locally and make sure to do the following.

You can use this sample [file](https://opensearch.org/docs/latest/install-and-configure/install-opensearch/docker/#sample-docker-compose-file-for-development) as an example.
Once your `docker-compose.yml` file is created, run the following command to start OpenSearch in the background:

```
docker-compose up -d
```


### b. Update cluster settings
### Update cluster settings

Ensure your cluster is configured to allow registering models. You can do this by updating the cluster settings using the following request:

Expand Down Expand Up @@ -146,7 +137,7 @@ POST /_plugins/_ml/models/_register
"passage_prefix": "passage: ",
"all_config": "{ \"_name_or_path\": \"intfloat/multilingual-e5-small\", \"architectures\": [ \"BertModel\" ], \"attention_probs_dropout_prob\": 0.1, \"hidden_size\": 384, \"num_attention_heads\": 12, \"num_hidden_layers\": 12, \"tokenizer_class\": \"XLMRobertaTokenizer\" }"
},
"url": "http://host.docker.internal:8080/intfloat-multilingual-e5-small-onnx.zip"
"url": "http://localhost:8080/intfloat-multilingual-e5-small-onnx.zip"
}
```

Expand Down Expand Up @@ -328,7 +319,7 @@ PUT _ingest/pipeline/asymmetric_embedding_ingest_pipeline

### 2.3 Simulate the pipeline

Simulate the pipeline by running the following request:
You can test the pipeline using the simulate endpoint. Simulate the pipeline by running the following request:
```
POST /_ingest/pipeline/asymmetric_embedding_ingest_pipeline/_simulate
{
Expand Down
Loading