Discrawl is a high-performance command-line utility designed to mirror Discord guilds into a local SQLite database, enabling offline search, analysis, and archiving of server history. Created to solve the limitations of Discord's native search functionality, Discrawl provides a robust way for organizations and individuals to maintain a local, searchable memory of their Discord presence without ongoing reliance on cloud-based search APIs. The tool is designed to work efficiently with either a dedicated Discord bot token for live synchronization or via local Git-backed snapshots for token-free offline access.

Discrawl mirrors Discord data into a structured SQLite database, which allows users to perform deep analytics, grepping, and SQL queries against their server history. It includes a comprehensive suite of features ranging from FTS5 lexical indexing to optional semantic search via vector embeddings. By supporting multiple sync sources—including direct bot API integration and local Discord Desktop cache wiretapping—Discrawl ensures that users can capture message data, member snapshots, and attachment metadata effectively while keeping sensitive operational details contained within their local environment.

Some of the key features are:

• Guild Mirroring: Automatically discovers and synchronizes channels, threads, members, and message history into a single, queryable SQLite database.
• FTS5 Search: Provides high-speed literal text search using the unicode61 tokenizer for precise results across thousands of messages.
• Desktop Wiretap: Imports messages and DMs directly from the local Discord Desktop cache without requiring a bot token or selfbot interactions.
• Semantic Search: Offers optional integration with providers like OpenAI or Ollama to generate vector embeddings for hybrid search capabilities.
• Git-backed Snapshots: Allows publishing archives as sharded NDJSON snapshots to private Git repositories for easy sharing and offline updates.
• Live Tail: Maintains an up-to-date archive by listening to the Discord Gateway, with automatic repair sweeps to ensure data consistency.
• Offline Analytics: Includes built-in commands for generating digests, trend analytics, and member reports using read-only SQL queries.
• Privacy-First Design: Sensitive data remains local, and sensitive internal diagnostic logs are excluded from published snapshots.

Operationally, Discrawl is a single Go binary that follows OS-native storage conventions. Users initialize the tool using the init command, configure their source (bot token or Git-snapshot), and then use sync to pull data. The doctor command validates the configuration and connectivity, while the tui command provides a terminal-based interface for browsing the archive. Data updates can be managed through manual syncs or continuous live tails. Users needing to extend the tool's capabilities can rely on read-only SQL access for custom reports or integrations with other tools.

Some common use cases include:

• Long-term Archive: Creating a permanent, searchable record of internal team communication that survives after channels or messages are deleted from Discord.
• Offline Research: Allowing researchers to perform complex linguistic or sociological analysis on server history without needing internet access.
• Security Auditing: Enabling security teams to query historical message and metadata patterns to identify potential policy violations or configuration issues.
• Enhanced Search: Providing a search interface that handles large volumes of text more efficiently than Discord's native client, with support for advanced lexical and semantic matching.