HardwareReport: Add OpenAI Assistant to Hardware Report Tool (Chat + Action Support)

[AnasChebili]

This PR adds an AI assistant chat interface to the Hardware Report Tool as part of the broader effort to enhance ArduPilot's web tools with intelligent assistant capabilities.

The assistant is capable of answering general questions about the webtool, answering questions about specific user-uploaded logs files, and performing predefined actions.

Currently, this requires using an OpenAI API key to use.

Below is a video showcasing a typical interaction:

assistant-showcase.mp4

Assistant Capabilities:

• Answers general questions about the tool from its instruction set.

• Analyzes uploaded log files using file search + code interpreter.

• Can call 3 functions: save_all_parameters, save_changed_parameters, save_minimal_parameters.

• Fully manages assistant lifecycle:

• Reuses or creates assistant based on API key.

• Upgrade button replaces old assistant with latest version.

• Auto-updates assistant context after the user changes the updated logs file.

• Code interpreter filters and processes large datasets efficiently. Big potential for improvement here.

Known Issues:

• Sometimes the assistant wrongly thinks a log file was (or wasn’t) uploaded. Happens rarely and usually goes away after a page reload and context reset.

• Not tested on large or complex real-world log files, might struggle there.

• Occasionally fails to retrieve data due to schema mismatch, causing errors in code interpreter. Asking it to try again usually fixes it.

• Needs more testing to surface other edge cases.

[Copilot]

Pull Request Overview

This PR integrates an OpenAI-based AI assistant into the Hardware Report Tool, providing chat capabilities, log-file analysis, and action functions.

• Adds schema and usage notes for log parsing (logs_schema_and_notes.txt)
• Defines system instructions for the assistant (instructions.txt)
• Embeds a chat UI and client logic (index.html, Chat.js)
• Declares assistant tool definitions (assistantTools.json)

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
logs_schema_and_notes.txt	Adds full JSON schema and processing notes for log segments
instructions.txt	Introduces assistant instructions and file‐presence guard logic
index.html	Inserts AI chat widget markup, styles, and script reference
assistantTools.json	Lists available assistant tools and function signatures
Chat.js	Implements assistant connection, file upload, threading, and UI

Comments suppressed due to low confidence (1)

HardwareReport/index.html:43

• The chat bubble container lacks an aria-label or role attribute; add aria-label="Open AI assistant chat" and role="button" to improve screen reader support.

  #ai-chat-bubble {

[AnasChebili]

I've addressed all comments. Regarding the API key: it's now stored in memory instead of localStorage, it will no longer persist and the user will have to re-enter it when page is refreshed, this is slightly better for security but still carries XSS risks.

[rmackay9]

Hi @AnasChebili,

Thanks very much for the updates. Feel free to mark the comments as resolved. I know there's sometimes a question, "is it ok for the dev to mark them as resolved or should the reporter check and then mark them him/herself" but in general we let the dev do it.

Thanks!

[AnasChebili]

@rmackay9 Done! Thanks!!

[IamPete1]

I get a error if I enter text without setting a API key.

It should prevent entering text until the key is provided.

The "thinking" box does not auto scroll the chat.

I started talking to the asistant before loading a log, now it cant see it:

In fact, I can't get it to "see" the log at all.

The other thing to check is that this works offline still. of course the chat won't work, but there should be no error and the reset of the tool should work as normal. Maybe the chat icon only turns up once the package has been loaded from the CDN.

Also, for me its not remembering the API key.

[AnasChebili]

Hey, thanks for the feedback!
I fixed the API key issue. Now if someone tries to chat without a saved or valid key, it shows an assistant error in the chat instead of throwing alerts. Also made the "thinking" box auto-scroll properly.
The tool still works fine offline. I added a check for the CDN package on load, and if it’s missing, the assistant bubble stays hidden.
For API key persistence, we removed local storage earlier in this PR due to security concerns. It’s just kept in memory now. Still not perfect, so Randy and I talked about adding backend middleware to handle it more securely. Would love your thoughts on that.
As for the assistant not seeing the log, yeah, it sometimes hallucinates whether a file was uploaded. The file is getting passed correctly though. I updated the instruction set so it always tries to use it and only shows an error if it actually fails. That seems more reliable than pre-checking.
Let me know if anything else feels off