Skip to content

t1c1/safe-historical-search

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

11 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Safe Historical Search

Safe Historical Search

๐Ÿ” Lightning-fast local search for ChatGPT and Claude conversation history

Export search tool โ€ข AI chat history โ€ข Privacy-first โ€ข Zero-cloud โ€ข Sub-second results โ€ข Interactive filtering

Version License Python Flask ChatGPT Claude


๐Ÿš€ Get Started in Under 60 Seconds

Transform your ChatGPT and Claude conversations into a searchable knowledge base instantly:

Keywords: ChatGPT export search, Claude conversation search, AI chat history, OpenAI export tool, Anthropic data search, conversation archive, AI knowledge base

๐Ÿ’ก Works with both Claude (Anthropic) and ChatGPT (OpenAI) exports! Mix multiple accounts from both providers seamlessly.

Step 1: Export Your AI Conversations

๐Ÿ“ค For Claude (Anthropic):

  1. Visit Claude Data Export
  2. Request data export โ†’ Wait for email โ†’ Download ZIP
  3. Extract: conversations.json, projects.json, users.json

๐Ÿ“ค For ChatGPT (OpenAI):

  1. Visit ChatGPT Data Controls
  2. Export data โ†’ Wait for email โ†’ Download ZIP
  3. Extract: conversations.json, user.json, shared_conversations.json, message_feedback.json

Step 2: One-Command Setup

git clone https://github.com/t1c1/safe-historical-search.git
cd safe-historical-search
mkdir -p files
# ๐Ÿ“ Copy your export files into ./files/
./quickstart.sh

๐ŸŒ Opens automatically at http://127.0.0.1:5001


โœจ What Makes This Special

๐ŸŽฏ Interactive Search Experience

  • Click any pill to filter instantly - Provider, role, or date
  • Real-time results - No page refreshes, instant filtering
  • Smart expand by default - Finds partial matches automatically
  • Native date pickers - Intuitive calendar widgets, not confusing sliders

๐Ÿ” Conversation Context

  • "Show context" on every result - See the full conversation thread
  • Highlighted current message - Know exactly where you are
  • Color-coded by role - Blue for human, green for AI responses
  • Expandable inline - No popups or new windows

๐Ÿ›ก๏ธ Privacy & Security

  • 100% local processing - Your data never leaves your machine
  • Zero cloud dependencies - Works completely offline
  • No tracking or analytics - No cookies, no external requests
  • Instant deletion - Remove the index folder to wipe everything

โšก Performance

  • Sub-second search - SQLite FTS5 full-text indexing
  • Handles thousands of conversations - Scales beautifully
  • Minimal resource usage - <50MB RAM typical
  • Cross-platform - macOS, Linux, Windows

๐ŸŽฏ Real-World Use Cases

๐Ÿ”ฌ Research & Knowledge Management

Perfect for researchers, consultants, data scientists, and knowledge workers who accumulate valuable insights through ChatGPT and Claude conversations. Search your AI chat exports like a personal research database.

Example workflows:

  • "Find that Python async solution from last month" โ†’ Search async python, filter by Assistant
  • "All my machine learning discussions" โ†’ Search machine learning, expand context to see evolution
  • "Competitor analysis from Q3" โ†’ Date filter + search competitor, see full strategic discussions

๐Ÿ’ผ Business Intelligence & Strategy

Extract patterns and insights from your AI brainstorming sessions.

Power user scenarios:

  • Market research synthesis - Find all conversations about industry trends
  • Product development tracking - See how feature ideas evolved over time
  • Decision documentation - Locate the reasoning behind key business choices
  • Team knowledge sharing - Search across multiple team members' exports

๐Ÿ“š Learning & Education

Students and lifelong learners can quickly retrieve educational content and build on previous learning.

Study workflows:

  • Concept reinforcement - Find previous explanations of complex topics
  • Problem-solving patterns - See how similar problems were approached
  • Study session continuity - Pick up where previous learning sessions left off
  • Cross-reference learning - Connect related concepts across different conversations

๐Ÿ› ๏ธ Developer & Technical Troubleshooting

Software developers, DevOps engineers, and IT professionals can build a personal knowledge base of coding solutions from their ChatGPT and Claude conversations. Perfect for searching code snippets, debugging solutions, and technical explanations.

Developer scenarios:

  • Bug fix retrieval - "How did I solve that Docker networking issue?"
  • Code pattern library - Find reusable solutions and code snippets
  • Architecture decisions - Review the reasoning behind technical choices
  • Learning documentation - Track your skill development over time

๐ŸŽจ Creative Project Management

Writers, designers, and creators can rediscover and build on creative ideas.

Creative workflows:

  • Idea archaeology - Rediscover forgotten creative concepts
  • Project evolution tracking - See how creative projects developed
  • Inspiration mining - Find sparks of creativity from past conversations
  • Collaboration history - Review feedback and iteration cycles

๐Ÿ”ง Advanced Features

๐ŸŽ›๏ธ Filtering & Search

  • Multi-provider support - Claude and ChatGPT in one interface
  • Role-based filtering - Human questions vs AI responses
  • Date range selection - Find conversations from specific time periods
  • Smart keyword expansion - Automatic wildcard matching for better recall
  • Relevance ranking - Most relevant results first, or sort by date

๐Ÿ“Š Data Management

  • Multi-account indexing - Combine multiple AI accounts seamlessly
  • Incremental updates - Easy reindexing with new conversation exports
  • Account separation - Each export source labeled and filterable
  • Conversation linking - Direct links back to original AI platforms
  • Export preservation - Original files remain untouched

๐Ÿ–ฅ๏ธ User Experience

  • Mobile responsive - Works great on phones and tablets
  • Keyboard navigation - Tab through results, shortcuts for power users
  • URL bookmarking - Share or save specific searches and filters
  • Loading states - Clear feedback during operations
  • Error handling - Graceful degradation with helpful error messages

โš™๏ธ Installation Options

๐Ÿš€ Quick Start (Recommended)

git clone https://github.com/t1c1/safe-historical-search.git
cd safe-historical-search
./quickstart.sh

Handles everything automatically - virtual environment, dependencies, indexing, and server startup.

๐Ÿ”ง Manual Installation

Click to expand step-by-step manual setup
# Clone repository
git clone https://github.com/t1c1/safe-historical-search.git
cd safe-historical-search

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Prepare data
mkdir -p ./files
# Copy your AI export files into ./files/

# Build search index
python index.py --export ./files --out ./index

# Start server
python server.py --db ./index/chatgpt.db --port 5001

๐Ÿณ Docker Setup

Coming in v0.3.0 - Docker containerization
# Future Docker support
docker run -v ./files:/data -v ./index:/index -p 5001:5001 safe-historical-search

๐ŸŽจ Search Tips & Power User Guide

๐Ÿ” Search Operators

  • Exact phrases: "machine learning model"
  • Wildcards: python* (finds python, pythonic, etc.)
  • Multiple terms: docker kubernetes deployment (finds all)
  • Date filtering: Use date pickers or URL params ?date_from=2024-01-01

โŒจ๏ธ Keyboard Shortcuts

  • Tab: Navigate through results
  • Enter: Expand conversation context
  • Ctrl+F: Focus search box
  • Escape: Clear current search

๐ŸŽฏ Pro Tips

  • Start broad, then narrow - Use general terms, then click pills to filter
  • Use context expansion - See how conversations evolved over time
  • Bookmark useful searches - URLs contain all filter parameters
  • Combine filters - Provider + Role + Date for laser-focused results
  • Check "Show context" - Often the full conversation has more valuable info

๐Ÿ›ก๏ธ Privacy & Security Deep Dive

๐Ÿ”’ Data Protection

  • Local-only processing - No internet required after setup
  • No telemetry - Zero data collection or transmission
  • Encrypted at rest - Your conversations stay in SQLite on your machine
  • Easy cleanup - Delete the ./index folder to remove all processed data
  • Source preservation - Original export files remain unchanged

๐Ÿ› ๏ธ Security Features

  • No external dependencies - Minimal attack surface
  • Open source - Full code transparency
  • No user accounts - No authentication or user tracking
  • Local web server - Only accessible from your machine by default
  • HTTPS ready - Can be configured with SSL certificates if needed

๐Ÿ” Data Handling

  • Conversation parsing - Extracts text content only, preserves structure
  • Metadata retention - Keeps timestamps, roles, and conversation IDs
  • No content modification - Your original conversations are never altered
  • Selective indexing - Choose which conversations to include

๐Ÿ—บ๏ธ Development Roadmap

โœ… Completed in v0.2.x

Originally planned for v0.2.0:

  • ๐Ÿค– ChatGPT (OpenAI) logs import โœ… Full ChatGPT export support implemented
  • ๐Ÿท๏ธ Account filtering in search UI โœ… Multi-account support with provider pills
  • ๐Ÿ“ฑ Single-step installer โœ… quickstart.sh provides one-command setup

Additional features delivered:

  • ๐ŸŽฏ Interactive clickable filters โœ… Click pills to filter instantly
  • ๐Ÿ” Expandable conversation context โœ… Full thread preview with highlighting
  • ๐Ÿ“… Smart date picker โœ… Native calendar widgets replace confusing sliders
  • ๐Ÿค– Unified AI roles โœ… Simplified to Human/Assistant (system+assistant combined)
  • โšก Real-time filtering โœ… Instant results without page refreshes
  • ๐Ÿ“ฑ Mobile responsiveness โœ… Works beautifully on all devices

๐ŸŽฏ Next Release (v0.3.0) - Q4 2025

  • ๐Ÿ” Advanced search operators - AND, OR, NOT, parentheses, quotes
  • ๐Ÿ’พ Saved searches - Bookmark and organize frequent searches
  • โŒจ๏ธ Keyboard shortcuts - Power user navigation and search
  • ๐Ÿ“Š Conversation statistics - Word counts, response times, topic trends
  • ๐Ÿ“ฑ Single-step installer - Auto-dependency management and setup
  • ๐Ÿ“ค Export functionality - Save filtered results to various formats

๐Ÿ”ฎ Future Releases (2026+)

  • โšก Incremental indexing - Auto-update with file watcher for new conversations
  • ๐Ÿง  Semantic search - AI-powered similarity search beyond keyword matching
  • ๐Ÿ“Š Export analytics - Advanced conversation insights and usage patterns
  • ๐Ÿท๏ธ Smart tagging - Auto-categorization and custom labels
  • ๐Ÿ”Œ Plugin ecosystem - Custom data sources and integrations
  • ๐ŸŒ Team collaboration - Multi-user deployments with privacy controls

๐Ÿ› ๏ธ Technical Architecture

๐Ÿ—๏ธ Core Components

  • Backend: Python 3.8+ with Flask web framework
  • Database: SQLite with FTS5 full-text search extension
  • Frontend: Vanilla JavaScript with modern CSS Grid/Flexbox
  • Search Engine: Optimized SQL queries with snippet highlighting
  • Data Processing: JSON parsing with robust error handling

โšก Performance Specifications

  • Index size: ~10-20MB per 1,000 conversations
  • Search latency: <100ms for most queries, <500ms for complex filters
  • Memory footprint: <50MB typical usage, <200MB with large datasets
  • Startup time: <2 seconds for databases up to 100,000 messages
  • Concurrent users: Supports multiple browser tabs, single-user focused

๐Ÿ“ File Format Support

  • Claude exports: conversations.json, projects.json, users.json
  • ChatGPT exports: conversations.json, user.json, shared_conversations.json, message_feedback.json
  • Mixed providers: Automatic detection and unified indexing
  • Version compatibility: Handles export format changes gracefully

๐Ÿ”ง Configuration Options

  • Custom ports: --port 8080 for different port binding
  • Database location: --db /path/to/custom.db for custom storage
  • Host binding: --host 0.0.0.0 for network access (use with caution)
  • Environment variables: CLAUDE_URL_TEMPLATE for custom linking
  • Index optimization: Configurable FTS5 parameters for performance tuning

๐Ÿค Contributing & Community

๐Ÿ› Bug Reports & Feature Requests

Found an issue or have an idea? We'd love to hear from you:

๐Ÿ’ป Contributing Code

  • Fork the repository and create a feature branch
  • Follow the existing code style - Python PEP 8, clean HTML/CSS/JS
  • Add tests for new functionality
  • Update documentation including README and inline comments
  • Submit a pull request with a clear description of changes

๐Ÿ“ Documentation

  • Wiki contributions - Help expand the documentation
  • Tutorial creation - Share your workflows and use cases
  • Translation - Help make this accessible to more users

๐Ÿ“‹ Version History

v0.2.1 - Current Release โœจ

  • Interactive filtering with clickable pills
  • Expandable conversation context
  • Smart date picker with native calendar widgets
  • Unified Human/Assistant role system
  • Real-time search updates
  • Mobile-responsive design improvements
  • Enhanced error handling and user feedback

v0.1.0 - Initial Release

  • Basic full-text search functionality
  • Claude and ChatGPT export support
  • Simple web interface
  • SQLite FTS5 indexing
  • Conversation linking

Coming Soon

See CHANGELOG.md for detailed release notes and migration guides.


๐Ÿ† Why Choose Safe Historical Search?

๐Ÿ†š vs. Manual File Searching

  • Instant results instead of opening dozens of files
  • Cross-conversation search finds related discussions
  • Visual context shows conversation flow and relationships
  • Smart filtering narrows down thousands of conversations instantly

๐Ÿ†š vs. Cloud-Based Solutions

  • Complete privacy - your conversations never leave your device
  • No subscription fees - free and open source forever
  • Offline capable - works without internet connection
  • No data limits - index unlimited conversations

๐Ÿ†š vs. Basic Search Tools

  • AI-conversation optimized - understands dialogue structure
  • Provider integration - links back to original conversations
  • Role-aware filtering - separate human questions from AI responses
  • Context preservation - see how ideas developed over time

๐ŸŒŸ Ready to Transform Your AI Conversations?

โญ Star this repository if it helps you unlock the knowledge in your AI conversations!

git clone https://github.com/t1c1/safe-historical-search.git
cd safe-historical-search && ./quickstart.sh

Built with โค๏ธ for AI conversation enthusiasts

Privacy-first โ€ข Open source โ€ข Made in America โ€ข Zero telemetry

๐Ÿ“– Documentation โ€ข ๐Ÿ› Issues โ€ข ๐Ÿ’ฌ Discussions โ€ข ๐Ÿ”„ Changelog

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published