Deep Thinking Tools

[TomLucidor]

Recently I came across this, thinking maybe TLDW can integrate it into creating new knowledge https://github.com/zjunlp/OmniThink

The key feature of deep thinking:

• Can search online information, including blogs, articles, news, audio, books, papers etc. in multiple formats
• can archive/cache online information well into a sematic DB, and chunk them at different resolution
• Can cross-link different pieces of information, and then add to a personal PKM
• The PKM would be fed back into the semantic DB for further thinking and research

[rmusser01]

Thanks for sharing the ideas, this is already on the roadmap(not that particular tool, but all those functions excluding PKM cross-linking) for the 'Researcher' module.

The goal is to allow for self-directed research up to a certain point; i.e. create X topics off of starting topic, proceed with Y methodology, using X tooling (existing as functionality available to the pipeline from tldw).

You can see the existing links I've collected for the feature under Docs/Researcher.md

[TomLucidor]

The problem would be to create an open source basket of "Y methodologies and Z tooling" such that the LLM can either try every human-grokked toolsets, or it can "freehand" and learn how to invent new tools of thoughts for humans tom use, but then human feedback would be needed to "grade" each method such that the LLM can further understand what works and what doesn't.

Three options on the "Deep Thinking" side:

• Pre-capture existing tools so the LLM can learn how to use them https://juuzt.ai/knowledge-base/prompt-frameworks/
• Mix in human feedback to see which frameworks are complete bunk, and which others are context-specific
• Baiting LLMs to experiment with creating new frameworks such that the good ones can be captured
• LLMs should be able to learn how to grade similar to a human, make it such that semi-supervised + RL are possible

Thanks for the notes, when it comes to experimental research rather than deep thoughts, it belongs to a separate discussion https://github.com/rmusser01/tldw/blob/main/Docs/Design/Researcher.md

[TomLucidor]

I need to drop this here just in case open-webui/open-webui#9321