From 09dd56c399fdc558a32547d72b0c53f5834fc96e Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Wed, 18 May 2022 07:44:37 -0700 Subject: [PATCH] Add authentication methods for import model script (#466) --- docs/guide/machine-learning.asciidoc | 53 ++++++++++++++++++++++++---- 1 file changed, 46 insertions(+), 7 deletions(-) diff --git a/docs/guide/machine-learning.asciidoc b/docs/guide/machine-learning.asciidoc index 2c2c6a8..57e52f6 100644 --- a/docs/guide/machine-learning.asciidoc +++ b/docs/guide/machine-learning.asciidoc @@ -41,16 +41,21 @@ model in {es}. For NLP tasks, Eland enables you to import PyTorch trained BERT models into {es}. Models can be either plain PyTorch models, or supported https://huggingface.co/transformers[transformers] models from the -https://huggingface.co/models[Hugging Face model hub]. +https://huggingface.co/models[Hugging Face model hub]. For example: [source,bash] ------------------------ -$ eland_import_hub_model \ - --url http://localhost:9200/ \ - --hub-model-id elastic/distilbert-base-cased-finetuned-conll03-english \ - --task-type ner \ +$ eland_import_hub_model \ <1> + --url http://localhost:9200/ \ <2> + --hub-model-id elastic/distilbert-base-cased-finetuned-conll03-english \ <3> + --task-type ner \ <4> --start ------------------------ +<1> Use an authentication method to access your cluster. Refer to <>. +<2> The cluster URL. Alternatively, use `--cloud-id`. +<3> Specify the identifier for the model in the Hugging Face model hub. +<4> Specify the type of NLP task. Supported values are `fill_mask`, `ner`, +`text_classification`, `text_embedding`, and `zero_shot_classification`. [source,python] ------------------------ @@ -67,7 +72,7 @@ Downloading: 100%|██████████| 208k/208k [00:00<00:00, 668kB/ Downloading: 100%|██████████| 112/112 [00:00<00:00, 43.9kB/s] Downloading: 100%|██████████| 249M/249M [00:23<00:00, 11.2MB/s] -# Export the model in a TorchScrpt representation which Elasticsearch uses +# Export the model in a TorchScript representation which Elasticsearch uses >>> tmp_path = "models" >>> Path(tmp_path).mkdir(parents=True, exist_ok=True) >>> model_path, config_path, vocab_path = tm.save(tmp_path) @@ -77,4 +82,38 @@ Downloading: 100%|██████████| 249M/249M [00:23<00:00, 11.2MB >>> ptm = PyTorchModel(es, tm.elasticsearch_model_id()) >>> ptm.import_model(model_path, config_path, vocab_path) 100%|██████████| 63/63 [00:12<00:00, 5.02it/s] ------------------------- \ No newline at end of file +------------------------ + +[discrete] +[[ml-nlp-pytorch-auth]] +==== Authentication methods + +The following authentication options are available when using the import script: + +* username and password authentication (specified with the `-u` and `-p` options): ++ +-- +[source,bash] +-------------------------------------------------- +eland_import_hub_model -u -p --cloud-id ... +-------------------------------------------------- +These `-u` and `-p` options also work when you use `--url`. +-- + +* username and password authentication (embedded in the URL): ++ +-- +[source,bash] +-------------------------------------------------- +eland_import_hub_model --url https://:@: ... +-------------------------------------------------- +-- + +* API key authentication: ++ +-- +[source,bash] +-------------------------------------------------- +eland_import_hub_model --es-api-key --url https://: ... +-------------------------------------------------- +--