fastNLP.models.bert module

fastNLP提供了BERT应用到五个下游任务的模型代码,可以直接调用。这五个任务分别为

每一个模型必须要传入一个名字为 embedfastNLP.embeddings.BertEmbedding ,这个参数包含了 fastNLP.modules.encoder.BertModel ,是下游模型的编码器(encoder)。

除此以外,还需要传入一个数字,这个数字在不同下游任务模型上的意义如下:

下游任务模型                     参数名称      含义
BertForSequenceClassification  num_labels  文本分类类别数目,默认值为2
BertForSentenceMatching        num_labels  Matching任务类别数目,默认值为2
BertForMultipleChoice          num_choices 多选任务选项数目,默认值为2
BertForTokenClassification     num_labels  序列标注标签数目,无默认值
BertForQuestionAnswering       num_labels  抽取式QA列数,默认值为2(即第一列为start_span, 第二列为end_span)

最后还可以传入dropout的大小,默认值为0.1。

class fastNLP.models.bert.BertForSequenceClassification(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels: int = 2, dropout=0.1)[源代码]

基类:fastNLP.models.base_model.BaseModel

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForSequenceClassification fastNLP.models.bert.BertForSequenceClassification

BERT model for classification.

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels: int = 2, dropout=0.1)[源代码]
参数
  • embed (fastNLP.embeddings.BertEmbedding) – 下游模型的编码器(encoder).

  • num_labels (int) – 文本分类类别数目,默认值为2.

  • dropout (float) – dropout的大小,默认值为0.1.

forward(words)[源代码]

输入为 [[w1, w2, w3, …], […]], BERTEmbedding会在开头和结尾额外加入[CLS]与[SEP] :param torch.LongTensor words: [batch_size, seq_len] :return: { fastNLP.Const.OUTPUT : logits}: torch.Tensor [batch_size, num_labels]

predict(words)[源代码]
参数

words (torch.LongTensor) – [batch_size, seq_len]

返回

{ fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size]

training: bool
class fastNLP.models.bert.BertForSentenceMatching(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels: int = 2, dropout=0.1)[源代码]

基类:fastNLP.models.base_model.BaseModel

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForSentenceMatching fastNLP.models.bert.BertForSentenceMatching

BERT model for sentence matching.

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels: int = 2, dropout=0.1)[源代码]
参数
  • embed (fastNLP.embeddings.BertEmbedding) – 下游模型的编码器(encoder).

  • num_labels (int) – Matching任务类别数目,默认值为2.

  • dropout (float) – dropout的大小,默认值为0.1.

forward(words)[源代码]

输入words的格式为 [sent1] + [SEP] + [sent2](BertEmbedding会在开头加入[CLS]和在结尾加入[SEP]),输出为batch_size x num_labels

参数

words (torch.LongTensor) – [batch_size, seq_len]

返回

{ fastNLP.Const.OUTPUT : logits}: torch.Tensor [batch_size, num_labels]

predict(words)[源代码]
参数

words (torch.LongTensor) – [batch_size, seq_len]

返回

{ fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size]

training: bool
class fastNLP.models.bert.BertForMultipleChoice(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_choices=2, dropout=0.1)[源代码]

基类:fastNLP.models.base_model.BaseModel

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForMultipleChoice fastNLP.models.bert.BertForMultipleChoice

BERT model for multiple choice.

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_choices=2, dropout=0.1)[源代码]
参数
  • embed (fastNLP.embeddings.BertEmbedding) – 下游模型的编码器(encoder).

  • num_choices (int) – 多选任务选项数目,默认值为2.

  • dropout (float) – dropout的大小,默认值为0.1.

forward(words)[源代码]
参数

words (torch.LongTensor) – [batch_size, num_choices, seq_len]

返回

{ fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size, num_choices]

predict(words)[源代码]
参数

words (torch.LongTensor) – [batch_size, num_choices, seq_len]

返回

{ fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size]

training: bool
class fastNLP.models.bert.BertForTokenClassification(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels, dropout=0.1)[源代码]

基类:fastNLP.models.base_model.BaseModel

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForTokenClassification fastNLP.models.bert.BertForTokenClassification

BERT model for token classification.

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding, num_labels, dropout=0.1)[源代码]
参数
  • embed (fastNLP.embeddings.BertEmbedding) – 下游模型的编码器(encoder).

  • num_labels (int) – 序列标注标签数目,无默认值.

  • dropout (float) – dropout的大小,默认值为0.1.

forward(words)[源代码]
参数

words (torch.LongTensor) – [batch_size, seq_len]

返回

{ fastNLP.Const.OUTPUT : logits}: torch.Tensor [batch_size, seq_len, num_labels]

predict(words)[源代码]
参数

words (torch.LongTensor) – [batch_size, seq_len]

返回

{ fastNLP.Const.OUTPUT : logits}: torch.LongTensor [batch_size, seq_len]

training: bool
class fastNLP.models.bert.BertForQuestionAnswering(embed: fastNLP.embeddings.bert_embedding.BertEmbedding)[源代码]

基类:fastNLP.models.base_model.BaseModel

基类 fastNLP.models.BaseModel

别名 fastNLP.models.BertForQuestionAnswering fastNLP.models.bert.BertForQuestionAnswering

用于做Q&A的Bert模型,如果是Squad2.0请将BertEmbedding的include_cls_sep设置为True,Squad1.0或CMRC则设置为False

__init__(embed: fastNLP.embeddings.bert_embedding.BertEmbedding)[源代码]
参数
  • embed (fastNLP.embeddings.BertEmbedding) – 下游模型的编码器(encoder).

  • num_labels (int) – 抽取式QA列数,默认值为2(即第一列为start_span, 第二列为end_span).

forward(words)[源代码]
输入words为question + [SEP] + [paragraph],BERTEmbedding在之后会额外加入开头的[CLS]和结尾的[SEP]. note:

如果BERTEmbedding中include_cls_sep=True,则输出的start和end index相对输入words会增加一位;如果为BERTEmbedding中 include_cls_sep=False, 则输出start和end index的位置与输入words的顺序完全一致

参数

words (torch.LongTensor) – [batch_size, seq_len]

返回

一个包含num_labels个logit的dict,每一个logit的形状都是[batch_size, seq_len + 2]

training: bool