Skip to content

Add support for JetStream generative inference for all KerasHub LLMs #1863

Open
@mattdangerw

Description

@mattdangerw

JetStream is throughput optimized generation software for JAX and TPUs. We should add support for JetStream generation for all KerasHub generative models.

The most useful starting reference for us will be the max engine implementation here.

Metadata

Metadata

Assignees

Labels

team-createdIssues created by Keras Hub team as part of development roadmap.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions