Tutorial for using Asymmetric models #3258

brianf-aws · 2024-12-06T01:44:36Z

Description

This PR implements the tutorial required for local asymmetric model embeddings. In this specific tutorial it is done using a docker container, to help users take advantage of multi node clusters using ML Nodes.

Related Issues

Resolves #3255

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

After replicating the local model embeddings. I am able to provide a high level solution of what the tutorial entails. Signed-off-by: Brian Flores <[email protected]>

Provides more context to each step Signed-off-by: Brian Flores <[email protected]>

Expands on the context of the previous commit and improvements in grammar and structure Signed-off-by: Brian Flores <[email protected]>

Signed-off-by: Brian Flores <[email protected]>

brianf-aws · 2024-12-06T23:06:34Z

There is a flaky test in the CI, can I get a retry please? :

org.opensearch.client.ResponseException: method [DELETE], host [http://127.0.0.1:33403/], URI [/_plugins/_ml/models/ooAcnpMBnp1fQxLKssuf], status line [HTTP/1.1 400 Bad Request]
    {"error":{"root_cause":[{"type":"status_exception","reason":"Model cannot be deleted in deploying or deployed state. Try undeploy model first then delete"}],"type":"status_exception","reason":"Model cannot be deleted in deploying or deployed state. Try undeploy model first then delete"},"status":400}

Zhangxunmt · 2024-12-06T23:06:56Z

docs/tutorials/semantic_search/asymmetric_embedding_model.md

+
+## Step 1: Spin Up a Docker OpenSearch Cluster
+
+To run OpenSearch in a local development environment, you can use Docker and a pre-configured `docker-compose` file.


This is not a requirement? I think the step 2-6 can also be done if you run an OpenSearch cluster locally without a docker?

Thats true, I chose a docker setup since there aren't many tutorials using it. Also its better with creating the tutorial with not having to register and deploy it again when I go back to use docker.

mingshl · 2024-12-06T23:30:04Z

can you please add the configuring the knn index using ml inference ingest processors, and also search using ml inference request processors? So we can give the full tutorials about how to use this model during ingest and search

zane-neo · 2024-12-10T01:34:23Z

docs/tutorials/semantic_search/asymmetric_embedding_model.md

+
+### b. Zip the Model Files
+
+In order to upload the model to OpenSearch, you must zip the necessary model files (`model.onnx`, `sentencepiece.bpe.model`, and `tokenizer.json`). The `model.onnx` file is located in the `onnx` directory of the cloned repository.


Is the onnx the only format that this model provided? Can we add pytorch format tutorial here?

Hey Zane! I can see why you ask Ill clarify that this is only for onnx, I havent used pytorch models so thats I wrote it like that

Got it, my suggestion is we can add both onnx and pytorch cases so that user can choose between them based on their cases.

Signed-off-by: Brian Flores <[email protected]>

brianf-aws · 2024-12-17T00:08:26Z

In order for this tutorial for all users following it, the following PR has to be merged to avoid the MLInput being null
#3281

kolchfa-aws

Some suggestions to clarify the text. In general, use sentence case capitalization and refer to the user as "you". Thanks!

docs/tutorials/semantic_search/asymmetric_embedding_model.md

kolchfa-aws · 2024-12-20T20:19:20Z

docs/tutorials/semantic_search/asymmetric_embedding_model.md

+This tutorial demonstrates how to generate text embeddings using an asymmetric embedding model in OpenSearch which will be used
+to run semantic search. This is implemented within a Docker container, the example model used in this tutorial is the multilingual
+`intfloat/multilingual-e5-small` model from Hugging Face. 
+You will learn how to prepare the model, register it in OpenSearch, and run inference to generate embeddings.


Suggested change

You will learn how to prepare the model, register it in OpenSearch, and run inference to generate embeddings.

In this tutorial, you'll learn how to prepare the model, register it in OpenSearch, and run inference to generate embeddings.

docs/tutorials/semantic_search/asymmetric_embedding_model.md

Signed-off-by: Brian Flores <[email protected]>

docs/tutorials/semantic_search/asymmetric_embedding_model.md

dhrubo-os · 2024-12-24T19:36:31Z

docs/tutorials/semantic_search/asymmetric_embedding_model.md

+
+To download the model, use the following steps:
+
+1. Install Git Large File Storage (LFS) if you haven’t already:


Seems like we should release this model from our pre-trained model repository so that customer can easily register this model?

What is the process to introduce the model (to the model repository)?

We can discuss offline more about this. But this is an example of the model tracing workflow from opensearch-py-ml

docs/tutorials/semantic_search/asymmetric_embedding_model.md

Signed-off-by: Brian Flores <[email protected]>

dhrubo-os · 2024-12-26T22:20:00Z

docs/tutorials/semantic_search/asymmetric_embedding_model.md

 ---

-## Step 1: Spin up a Docker OpenSearch cluster
+## Step 1: Start OpenSearch locally


Do we care if customer started opensearch locally or with public IP? In our blue print example, we setup OS locally.

Ideally locally because I have a step were I make them use python to service the model which runs on localhost:8080

if I have a OS which is running in let's say 8.8.8.8 and then if I use 8.8.8.8:8080, won't that work?

Not an expert but if two processes run on the same address like localhost but use different ports then it should be fine. Also the python server is just a one time occurrence just a means to get OS to download the model.

It should work. If we setup a OS cluster in EC2 host, we can access through the public URL and the port. So I would suggest to rephrase: Start Opensearch and in line 22 we can say:

Run OpenSearch and ensure the following steps are completed. In this example, we set up the OS cluster locally.

Makes sense, thank you for the suggestion (addressed in commit b11cbe1).

Signed-off-by: Brian Flores <[email protected]>

dhrubo-os · 2024-12-27T17:32:00Z

docs/tutorials/semantic_search/asymmetric_embedding_model.md

+
+### b. Zip the model files
+
+To upload the model to OpenSearch, you must zip the necessary model files (`model.onnx`, `sentencepiece.bpe.model`, and `tokenizer.json`). The `model.onnx` file is located in the `onnx` directory of the cloned repository.


all these three files are located in onnx directory, not just model.onnx

Addressed in latest commit! c43e9a5

dhrubo-os · 2024-12-27T17:37:24Z

docs/tutorials/semantic_search/asymmetric_embedding_model.md

+   git lfs install
+   ```
+
+2. Clone the model repository:


Rather than lfs install or cloning the whole repository may be we can do?

Create a Directory for the Files:

mkdir multilingual-e5-model cd multilingual-e5-model

Download the Files Using wget or curl: Use the raw file links from the Hugging Face repository.

wget https://huggingface.co/intfloat/multilingual-e5-small/resolve/main/onnx/model.onnx wget https://huggingface.co/intfloat/multilingual-e5-small/resolve/main/tokenizer.json wget https://huggingface.co/intfloat/multilingual-e5-small/resolve/main/sentencepiece.bpe.model

Alternatively, use curl:

curl -O https://huggingface.co/intfloat/multilingual-e5-small/resolve/main/onnx/model.onnx curl -O https://huggingface.co/intfloat/multilingual-e5-small/resolve/main/tokenizer.json curl -O https://huggingface.co/intfloat/multilingual-e5-small/resolve/main/sentencepiece.bpe.model

I hear you on this but lets suppose there is a new update on the model. user would have to delete and make a separate request. Also I think separate requests would cause issues since these are big files users are working on. Also its possible the endpoint can change or someone changes directories for a clean up. We lock ourselves into maintaining a tutorial for their changes

lets suppose there is a new update on the model. user would have to delete and make a separate request.

Yeah, if they want to use the new updated model, don't they need to do the same for your case to use the updated model? They need to pull the updates from the git.

its possible the endpoint can change or someone changes directories for a clean up. We lock ourselves into maintaining a tutorial for their changes

I think this is true for both cases, if they change the file name, we need to update the doc in both cases, right?

My thought process is, in this way:

customer doesn't need to install git & git lfs

customer doesn't need to download unnecessary package which they don't need anymore.

This is not a blocker, but a suggestion.

Signed-off-by: Brian Flores <[email protected]>

adds initial tutorial contents

f605636

After replicating the local model embeddings. I am able to provide a high level solution of what the tutorial entails. Signed-off-by: Brian Flores <[email protected]>

brianf-aws requested a deployment to ml-commons-cicd-env-require-approval December 6, 2024 01:44 — with GitHub Actions Waiting

brianf-aws added 3 commits December 6, 2024 11:34

Add: initial thoughts to highlevel steps

bcdd180

Provides more context to each step Signed-off-by: Brian Flores <[email protected]>

expand more details on tutorial

8cd8359

Expands on the context of the previous commit and improvements in grammar and structure Signed-off-by: Brian Flores <[email protected]>

fix typo

c792403

Signed-off-by: Brian Flores <[email protected]>

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 6, 2024 21:36 — with GitHub Actions Inactive

brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval December 6, 2024 21:36 — with GitHub Actions Failure

brianf-aws marked this pull request as ready for review December 6, 2024 21:36

brianf-aws requested review from b4sjoo, dhrubo-os, jngz-es, model-collapse, rbhavna, ylwu-amzn, zane-neo, Zhangxunmt, austintlee, HenryL27 and xinyual as code owners December 6, 2024 21:36

Zhangxunmt reviewed Dec 6, 2024

View reviewed changes

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 6, 2024 23:21 — with GitHub Actions Inactive

zane-neo reviewed Dec 10, 2024

View reviewed changes

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 10, 2024 07:09 — with GitHub Actions Inactive

brianf-aws mentioned this pull request Dec 13, 2024

[BUG] MLInferenceIngestProcessor has xContentRegistry as null #3276

Open

mingshl mentioned this pull request Dec 16, 2024

Add support for asymmetric embedding models opensearch-project/neural-search#710

Open

5 tasks

adds Asymmetric semantic search

d48f75c

Signed-off-by: Brian Flores <[email protected]>

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 17, 2024 00:02 — with GitHub Actions Inactive

brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval December 17, 2024 01:03 — with GitHub Actions Failure

kolchfa-aws reviewed Dec 20, 2024

View reviewed changes

Apply Code/Writing review

f4ac9a0

Signed-off-by: Brian Flores <[email protected]>

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 23, 2024 21:22 — with GitHub Actions Inactive

dhrubo-os reviewed Dec 24, 2024

View reviewed changes

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 24, 2024 20:34 — with GitHub Actions Inactive

dhrubo-os reviewed Dec 26, 2024

View reviewed changes

docs/tutorials/semantic_search/asymmetric_embedding_model.md Outdated Show resolved Hide resolved

Removed Docker requirement

47bf8ca

Signed-off-by: Brian Flores <[email protected]>

brianf-aws requested a deployment to ml-commons-cicd-env-require-approval December 26, 2024 21:59 — with GitHub Actions Waiting

dhrubo-os reviewed Dec 26, 2024

View reviewed changes

Make clear OpenSearch is ran locally in tutorial

b11cbe1

Signed-off-by: Brian Flores <[email protected]>

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 27, 2024 00:10 — with GitHub Actions Inactive

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 27, 2024 17:01 — with GitHub Actions Inactive

dhrubo-os reviewed Dec 27, 2024

View reviewed changes

Simplify explanation of file location

c43e9a5

Signed-off-by: Brian Flores <[email protected]>

brianf-aws had a problem deploying to ml-commons-cicd-env-require-approval December 27, 2024 22:28 — with GitHub Actions Failure

brianf-aws temporarily deployed to ml-commons-cicd-env-require-approval December 27, 2024 22:28 — with GitHub Actions Inactive

dhrubo-os mentioned this pull request Dec 27, 2024

[FEATURE]Release Asynmetric Model from Opensearch Model Server #3301

Open

dhrubo-os added the backport 2.x label Dec 27, 2024

dhrubo-os approved these changes Dec 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tutorial for using Asymmetric models #3258

Tutorial for using Asymmetric models #3258

brianf-aws commented Dec 6, 2024

brianf-aws commented Dec 6, 2024

Zhangxunmt Dec 6, 2024

brianf-aws Dec 6, 2024 •

edited

Loading

mingshl commented Dec 6, 2024

zane-neo Dec 10, 2024

brianf-aws Dec 10, 2024

zane-neo Dec 10, 2024

brianf-aws commented Dec 17, 2024

kolchfa-aws left a comment

kolchfa-aws Dec 20, 2024

dhrubo-os Dec 24, 2024

brianf-aws Dec 26, 2024

dhrubo-os Dec 26, 2024

dhrubo-os Dec 26, 2024

brianf-aws Dec 26, 2024

dhrubo-os Dec 26, 2024

brianf-aws Dec 26, 2024

dhrubo-os Dec 26, 2024

brianf-aws Dec 27, 2024

dhrubo-os Dec 27, 2024

brianf-aws Dec 27, 2024

dhrubo-os Dec 27, 2024

brianf-aws Dec 27, 2024

dhrubo-os Dec 27, 2024 •

edited

Loading


		## Step 1: Spin Up a Docker OpenSearch Cluster

		To run OpenSearch in a local development environment, you can use Docker and a pre-configured `docker-compose` file.


		### b. Zip the Model Files

		In order to upload the model to OpenSearch, you must zip the necessary model files (`model.onnx`, `sentencepiece.bpe.model`, and `tokenizer.json`). The `model.onnx` file is located in the `onnx` directory of the cloned repository.

	You will learn how to prepare the model, register it in OpenSearch, and run inference to generate embeddings.
	In this tutorial, you'll learn how to prepare the model, register it in OpenSearch, and run inference to generate embeddings.


		To download the model, use the following steps:

		1. Install Git Large File Storage (LFS) if you haven’t already:


		### b. Zip the model files

		To upload the model to OpenSearch, you must zip the necessary model files (`model.onnx`, `sentencepiece.bpe.model`, and `tokenizer.json`). The `model.onnx` file is located in the `onnx` directory of the cloned repository.

Tutorial for using Asymmetric models #3258

Are you sure you want to change the base?

Tutorial for using Asymmetric models #3258

Conversation

brianf-aws commented Dec 6, 2024

Description

Related Issues

Check List

brianf-aws commented Dec 6, 2024

Choose a reason for hiding this comment

brianf-aws Dec 6, 2024 • edited Loading

Choose a reason for hiding this comment

mingshl commented Dec 6, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianf-aws commented Dec 17, 2024

kolchfa-aws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dhrubo-os Dec 27, 2024 • edited Loading

Choose a reason for hiding this comment

brianf-aws Dec 6, 2024 •

edited

Loading

dhrubo-os Dec 27, 2024 •

edited

Loading