Chrome Prompt API doesn't work due to allow=language-model needed

[ryanseddon]

As discussed in #238 the Chrome (and most likely Edge too) Prompt API doesn't work in observable after Chrome made a change to need cross origin iframes to set allow="language-model".

I've tried permission proxy idea via a popup window to no avail in this notebook.

Trying to start a session gives me permission denied

[mootari]

What's your Chrome version? Mine is 138.0.7204.158 (Official Build) (arm64) and I can still access the API normally through a notebook.

LanguageModel.create().then(s => s.prompt("say hello"))

[ryanseddon]

Interesting so I didn't realise this was in stable, been working in dev and canary. I went through the fresh setup and downloaded the model and then I can get it to work with the proxy window (I didn't realise that I had to keep it open but it makes sense now why 🤦 )

I still can't get it to work in dev or canary where I had the model already downloaded and available

@tomayac any ideas why that would be?

[mootari]

Afaik you don't need the proxy window in 138, and for 139 it doesn't seem to be a suitable workaround because the permission can't be requested.

[tomayac]

Afaik you don't need the proxy window in 138, and for 139 it doesn't seem to be a suitable workaround because the permission can't be requested.

That's right, as the change was only introduced in 139. This proxy window approach won't work, unless you come up with an ugly workaround where the AI logic runs on the embedding page and then postMessage()es back and forth to the iframe. The allow attribute is the way to go here.

[tomayac]

Hi Oberservable HQ folks,

Tom from Chrome DevRel here. For built-in AI features, as currently supported by Edge and Chrome, to work again, please consider adding the following values to your allow attribute.

• language-model (with a dash)
• translator
• language-detector (with a dash)
• writer
• rewriter
• summarizer
• proofreader

You already support allow in your current implementation:

[tomayac]

Hey Tom, thanks for chiming in. Which version of Chrome introduced the change?

This change was introduced in this change, which made it into Chrome 139.

[mootari]

Are there any plans to make these requestable permissions? It seems unlikely to me that we'll just add them to the sandbox iframe's allow list given the impact they can have.

[tomayac]

There are no plans to gate these behind a permission, as you can do everything these APIs can do in userland JavaScript, including downloading a multi-gigabytes file (in general that is, not sure about the sandbox you put around JavaScript in ObservableHQ). These APIs just make working with AI features more ergonomic.

[mootari]

I don't have the bandwidth right now to look into the specifics, but posting this link as reference for later: https://github.com/webmachinelearning/prompt-api/blob/main/security-privacy-questionnaire.md

[tomayac]

Relevant could be 3.:

Do the features in your specification expose personal information, personally-identifiable information (PII), or information derived from either?

No. Although it's imaginable that the backing language model could be fine-tuned on PII to give more accurate-to-this-user outputs, we intend to disallow this in the specification.

The attack vector here with a built-in model (that is, not a downloaded per-origin model as in the userland solutions) would be that the user tells the model on health.example.com about their health situation, which, theoretically, the model could remember. An evil site evil.example.com could then trigger a hidden prompt to ask the model for the user's health situation. As outlined in the answer, this is something we don't do and intend to disallow in the specification, so valid implementations outside of Chrome can't add such a feature either.