You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have extended support for HuggingFace 🤗 and TF Hub exported models since 3.1.0 to equivalent Spark NLP 🚀 annotators. Starting this release, you can easily use the saved_model feature in HuggingFace within a few lines of codes and import any BERT, DistilBERT, CamemBERT, RoBERTa, DeBERTa, XLM-RoBERTa, Longformer, BertForTokenClassification, DistilBertForTokenClassification, AlbertForTokenClassification, RoBertaForTokenClassification, DeBertaForTokenClassification, XlmRoBertaForTokenClassification, XlnetForTokenClassification, LongformerForTokenClassification, CamemBertForTokenClassification, CamemBertForSequenceClassification, CamemBertForQuestionAnswering, BertForSequenceClassification, DistilBertForSequenceClassification, AlbertForSequenceClassification, RoBertaForSequenceClassification, DeBertaForSequenceClassification, XlmRoBertaForSequenceClassification, XlnetForSequenceClassification, LongformerForSequenceClassification, AlbertForQuestionAnswering, BertForQuestionAnswering, DeBertaForQuestionAnswering, DistilBertForQuestionAnswering, LongformerForQuestionAnswering, RoBertaForQuestionAnswering, XlmRoBertaForQuestionAnswering, TapasForQuestionAnswering, Vision Transformers (ViT), HubertForCTC, SwinForImageClassification, and ConvNextForImageClassification models to Spark NLP. We will work on the remaining annotators and extend this support to the rest with each release 😊
Compatibility
Spark NLP: The equivalent annotator in Spark NLP TF Hub: Models from TF Hub HuggingFace: Models from HuggingFace ONNX: Models from HuggingFace in ONNX format Model Architecture: Which architecture is compatible with that annotator Flags:
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Import Transformers into Spark NLP
Overview
We have extended support for
HuggingFace
🤗 andTF Hub
exported models since3.1.0
to equivalent Spark NLP 🚀 annotators. Starting this release, you can easily use thesaved_model
feature in HuggingFace within a few lines of codes and import anyBERT
,DistilBERT
,CamemBERT
,RoBERTa
,DeBERTa
,XLM-RoBERTa
,Longformer
,BertForTokenClassification
,DistilBertForTokenClassification
,AlbertForTokenClassification
,RoBertaForTokenClassification
,DeBertaForTokenClassification
,XlmRoBertaForTokenClassification
,XlnetForTokenClassification
,LongformerForTokenClassification
,CamemBertForTokenClassification
,CamemBertForSequenceClassification
,CamemBertForQuestionAnswering
,BertForSequenceClassification
,DistilBertForSequenceClassification
,AlbertForSequenceClassification
,RoBertaForSequenceClassification
,DeBertaForSequenceClassification
,XlmRoBertaForSequenceClassification
,XlnetForSequenceClassification
,LongformerForSequenceClassification
,AlbertForQuestionAnswering
,BertForQuestionAnswering
,DeBertaForQuestionAnswering
,DistilBertForQuestionAnswering
,LongformerForQuestionAnswering
,RoBertaForQuestionAnswering
,XlmRoBertaForQuestionAnswering
,TapasForQuestionAnswering
,Vision Transformers (ViT)
,HubertForCTC
,SwinForImageClassification
, andConvNextForImageClassification
models to Spark NLP. We will work on the remaining annotators and extend this support to the rest with each release 😊Compatibility
Spark NLP: The equivalent annotator in Spark NLP
TF Hub: Models from TF Hub
HuggingFace: Models from HuggingFace
ONNX: Models from HuggingFace in ONNX format
Model Architecture: Which architecture is compatible with that annotator
Flags:
Example Notebooks
HuggingFace, Optimum, PyTorch, and ONNX Runtime to Spark NLP (ONNX)
HuggingFace to Spark NLP (TensorFlow)
TF Hub to Spark NLP
Beta Was this translation helpful? Give feedback.
All reactions