-
Notifications
You must be signed in to change notification settings - Fork 1
feat: configurable examples #132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
freinold
wants to merge
8
commits into
main
Choose a base branch
from
feat-configurable-examples
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
d7d034e
feat(examples): add configurable example module
freinold 0546ef6
feat(examples): use examples as defaults in data model
freinold 70d4592
feat(examples): use examples in frontend
freinold a261184
chore: remove unused helper module
freinold 816920e
chore: move entity to package root to hinder circular imports
freinold 0b235d9
feat(examples): add additional example text and entities for better c…
freinold 0801d1a
feat(examples): add more diverse example texts and entity types for i…
freinold adc5dfc
fix(frontend): update example option selection logic for clarity
freinold File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
from pydantic import AliasChoices, BaseModel, Field | ||
|
||
|
||
class Entity(BaseModel): | ||
start: int = Field( | ||
ge=0, | ||
description="Start index of the entity in the input text", | ||
) | ||
end: int = Field( | ||
ge=0, | ||
description="End index of the entity in the input text", | ||
) | ||
text: str = Field( | ||
description="Text of the entity, extracted from the input text", | ||
) | ||
type: str = Field( | ||
validation_alias=AliasChoices("type", "label"), | ||
description="Entity type or label", | ||
) | ||
score: float = Field( | ||
ge=0.0, | ||
le=1.0, | ||
description="Confidence score of the entity detection, between 0 and 1", | ||
) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,127 @@ | ||
from functools import lru_cache | ||
|
||
from pydantic import Field | ||
from pydantic_settings import ( | ||
BaseSettings, | ||
JsonConfigSettingsSource, | ||
PydanticBaseSettingsSource, | ||
SettingsConfigDict, | ||
YamlConfigSettingsSource, | ||
) | ||
|
||
from gliner_api import Entity | ||
from gliner_api.config import Config, get_config | ||
|
||
config: Config = get_config() | ||
|
||
|
||
class Examples(BaseSettings): | ||
invoke: list["InvokeExample"] = Field( | ||
default=[ | ||
{ | ||
"text": "Steve Jobs founded Apple Inc. in Cupertino, CA on April 1, 1976.", | ||
"entities": [ | ||
Entity(start=0, end=10, text="Steve Jobs", type="person", score=0.99), | ||
Entity(start=19, end=24, text="Apple", type="organization", score=0.98), | ||
Entity(start=28, end=37, text="Cupertino", type="location", score=0.98), | ||
Entity(start=39, end=49, text="California", type="location", score=0.99), | ||
Entity(start=53, end=66, text="April 1, 1976", type="date", score=0.68), | ||
], | ||
}, | ||
{ | ||
"text": "Until her death in 2022, the head of the Windsor family, Queen Elizabeth, resided in London.", | ||
"entity_types": ["person", "organization", "location", "date"], | ||
"entities": [], | ||
}, | ||
{ | ||
"text": "The Eiffel Tower was completed in 1889 and is located in Paris, France.", | ||
"entity_types": ["building", "location", "date"], | ||
"entities": [], | ||
}, | ||
{ | ||
"text": "Barack Obama served as the 44th President of the United States from 2009 to 2017.", | ||
"threshold": 0.4, | ||
"entity_types": ["person", "organization", "location", "date", "job title"], | ||
"flat_ner": False, | ||
"entities": [], | ||
}, | ||
{ | ||
"text": "Albert Einstein developed the theory of relativity, which revolutionized modern physics.", | ||
"threshold": 0.2, | ||
"entity_types": ["person", "research field", "topic", "physical law"], | ||
"multi_label": True, | ||
"entities": [], | ||
}, | ||
] | ||
) | ||
batch: list["BatchExample"] = Field( | ||
default=[ | ||
{ | ||
"texts": [ | ||
"Steve Jobs founded Apple Inc. in Cupertino, CA on April 1, 1976.", | ||
"Until her death in 2022, the head of the Windsor family, Queen Elizabeth, resided in London.", | ||
], | ||
"entities": [ | ||
[ | ||
Entity(start=0, end=10, text="Steve Jobs", type="person", score=0.99), | ||
Entity(start=19, end=24, text="Apple", type="organization", score=0.98), | ||
Entity(start=28, end=37, text="Cupertino", type="location", score=0.98), | ||
Entity(start=39, end=49, text="California", type="location", score=0.99), | ||
Entity(start=53, end=66, text="April 1, 1976", type="date", score=0.68), | ||
], | ||
[ | ||
Entity(start=19, end=23, text="2022", type="date", score=0.38), | ||
Entity(start=41, end=55, text="Windsor family", type="organization", score=0.90), | ||
Entity(start=57, end=72, text="Queen Elizabeth", type="person", score=0.99), | ||
Entity(start=85, end=91, text="London", type="location", score=0.99), | ||
], | ||
], | ||
} | ||
] | ||
) | ||
|
||
model_config: SettingsConfigDict = SettingsConfigDict( | ||
yaml_file="examples.yaml", | ||
yaml_file_encoding="utf-8", | ||
json_file="examples.json", | ||
json_file_encoding="utf-8", | ||
) | ||
|
||
@classmethod | ||
def settings_customise_sources( | ||
cls, | ||
settings_cls: type[BaseSettings], | ||
init_settings: PydanticBaseSettingsSource, | ||
env_settings: PydanticBaseSettingsSource, | ||
dotenv_settings: PydanticBaseSettingsSource, | ||
file_secret_settings: PydanticBaseSettingsSource, | ||
) -> tuple[PydanticBaseSettingsSource, ...]: | ||
return ( | ||
init_settings, | ||
YamlConfigSettingsSource(settings_cls=settings_cls), | ||
JsonConfigSettingsSource(settings_cls=settings_cls), | ||
) | ||
|
||
|
||
class InvokeExample(BaseSettings): | ||
text: str | ||
threshold: float = Field(ge=0.0, le=1.0, default=config.default_threshold) | ||
entity_types: list[str] = Field(default=config.default_entities) | ||
flat_ner: bool = True | ||
multi_label: bool = False | ||
entities: list[Entity] | ||
|
||
|
||
class BatchExample(BaseSettings): | ||
texts: list[str] | ||
threshold: float = Field(ge=0.0, le=1.0, default=config.default_threshold) | ||
entity_types: list[str] = Field(default=config.default_entities) | ||
flat_ner: bool = True | ||
multi_label: bool = False | ||
entities: list[list[Entity]] | ||
|
||
|
||
@lru_cache | ||
def get_examples() -> Examples: | ||
"""Get the examples for the API docs and the frontend.""" | ||
return Examples() |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💡 Verification agent
🧩 Analysis chain
Verify entity start/end indices are accurate.
Please double-check that the entity boundaries are correct. For example:
🏁 Script executed:
Length of output: 578
Update entity indices to match the actual example text
After verifying the string
the spans for “Cupertino”, “CA”/“California” and the date are off. Please correct lines 24–28 in
gliner_api/examples.py
:📝 Committable suggestion
🤖 Prompt for AI Agents