Skip to content

Scope BertSpanLabeler and supporting infra #625

Open
@jbischof

Description

@jbischof

BertClassifier supports one output for the entire input sequence, but span labeling is an important research benchmark. This will require some scoping and design:

  1. Do we need a separate BertSpanLabelingPreprocessor or just a more complicated label to pass through?
  2. Do we want to offer a script that can turn a standard dataset like squadv2 into something our preprocessor can use? The "answer text" must be turned into indicies in the tokenized input.
  3. Create a BertSpanLabeler task model with the correct task-specific layers and preprocessing.

Metadata

Metadata

Assignees

Labels

scoping requiredFeatures that need significant design and planning before being actionabletype:featureNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions