Caikit embeddings examples + local run documentation #348
Conversation
flaviabeo
requested review from
Ssukriti,
alex-jw-brooks,
evaline-ju,
gabe-l-hart,
gkumbhat and
tharapalanivel
as code owners
flaviabeo
force-pushed
the
embeddings_docs
branch
from
643eeb9 to
bb7afe7
Compare
flaviabeo
force-pushed
the
embeddings_docs
branch
3 times, most recently
from
9484fa2 to
6391f73
Compare
flaviabeo
force-pushed
the
embeddings_docs
branch
from
6391f73 to
5e750cc
Compare
evaline-ju
left a comment
evaline-ju
left a comment
There was a problem hiding this comment.
Thanks for adding examples! Some suggestions + looks like formatting is failing
| "model_id": model_id | ||
| } | ||
| response = requests.post( | ||
| f"http://{host}:8080/api/v1/task/rerank-tasks", |
There was a problem hiding this comment.
The host variable is only present under the grpc section - we may want to move that variable out and not make this part conditional on grpc?
There was a problem hiding this comment.
removed this to another file - let me know if that was the suggestion?
There was a problem hiding this comment.
ah my suggestion wasn't particularly that this had to be a separate file. Previously, the host variable was only under the if get_config().runtime.grpc.enabled: so if that had been disabled, this http portion would've errored. As long as this code runs in all known cases and is easy to follow for a user, I am not particular on file placement 😄
flaviabeo
force-pushed
the
embeddings_docs
branch
from
3c04e7f to
0faa969
Compare
Signed-off-by: Flavia Beo <[email protected]>
flaviabeo
force-pushed
the
embeddings_docs
branch
from
0faa969 to
ded3b55
Compare
evaline-ju
left a comment
evaline-ju
left a comment
There was a problem hiding this comment.
LGTM with nit, assuming all the notebooks and scripts have been run again and confirmed to work with the recent changes
| "model_id": model_id | ||
| } | ||
| response = requests.post( | ||
| f"http://{host}:8080/api/v1/task/rerank-tasks", |
There was a problem hiding this comment.
ah my suggestion wasn't particularly that this had to be a separate file. Previously, the host variable was only under the if get_config().runtime.grpc.enabled: so if that had been disabled, this http portion would've errored. As long as this code runs in all known cases and is easy to follow for a user, I am not particular on file placement 😄
| request, metadata=[("mm-model-id", model_id)], timeout=1 | ||
| ) | ||
|
|
||
| # print("RESPONSE:", response) |
There was a problem hiding this comment.
nit: removable?
| # print("RESPONSE:", response) | |
| # print("RESPONSE:", response) |
| os.environ['ALLOW_DOWNLOADS'] = "1" | ||
|
|
||
| import caikit_nlp | ||
| model_name = "BAAI/bge-large-en-v1.5" |
There was a problem hiding this comment.
Can we change the model name to "sentence-transformers/all-MiniLM-L6-v2" just so that the person who follows this tutorial can mostly copy and paste the commands?
3 participants
No description provided.