7 Commits

Author SHA1 Message Date
Sarv
a745519451 message search (#60) 2025-04-19 03:06:05 +08:00
Sarv
85b5465d2a fix db file dependencies (#56) 2025-04-18 00:31:19 +08:00
Sarv
b866d6eddd fix server cmd & http index page (#54) 2025-04-17 18:50:09 +08:00
Sarv
871ad50b3b server command (#49) 2025-04-17 01:04:50 +08:00
Sarv
b64a3a1caa docs (#48) 2025-04-16 18:19:41 +08:00
Sarv
25d0b394e2 auto decrypt (#44) 2025-04-16 01:02:29 +08:00
Sarv
f2aa923e99 support voice message (#31) 2025-04-12 03:10:32 +08:00
51 changed files with 5452 additions and 1476 deletions

121
DISCLAIMER.md Normal file
View File

@@ -0,0 +1,121 @@
# Chatlog 免责声明
## 1. 定义
在本免责声明中,除非上下文另有说明,下列术语应具有以下含义:
- **"本项目"或"Chatlog"**:指本开源软件项目,包括其源代码、可执行程序、文档及相关资源。
- **"开发者"**:指本项目的创建者、维护者及代码贡献者。
- **"用户"**:指下载、安装、使用或以任何方式接触本项目的个人或实体。
- **"聊天数据"**:指通过各类即时通讯软件生成的对话内容及相关元数据。
- **"合法授权"**:指根据适用法律法规,由数据所有者或数据主体明确授予的处理其聊天数据的权限。
- **"第三方服务"**:指由非本项目开发者提供的外部服务,如大型语言模型(LLM) API 服务。
## 2. 使用目的与法律遵守
本项目仅供学习、研究和个人合法使用。用户须严格遵守所在国家/地区的法律法规使用本工具。任何违反法律法规、侵犯他人合法权益的行为,均与本项目及其开发者无关,相关法律责任由用户自行承担。
⚠️ **用户应自行了解并遵守当地有关数据访问、隐私保护、计算机安全和网络安全的法律法规。不同司法管辖区对数据处理有不同的法律要求,用户有责任确保其使用行为符合所有适用法规。**
## 3. 授权范围与隐私保护
- 本工具仅限于处理用户自己合法拥有的聊天数据,或已获得数据所有者明确授权的数据。
- 严禁将本工具用于未经授权获取、查看或分析他人聊天记录,或侵犯他人隐私权。
- 用户应采取适当措施保护通过本工具获取和处理的聊天数据安全,包括但不限于加密存储、限制访问权限、定期删除不必要数据等。
- 用户应确保其处理的聊天数据符合相关数据保护法规,包括但不限于获得必要的同意、保障数据主体权利、遵守数据最小化原则等。
## 4. 使用限制
- 本项目仅允许在合法授权情况下对聊天数据库进行备份与查看。
- 未经明确授权,严禁将本项目用于访问、查看、分析或处理任何第三方聊天数据。
- 使用第三方 LLM 服务时,用户应遵守相关服务提供商的服务条款和使用政策。
- 用户不得规避本项目中的任何技术限制,或尝试反向工程、反编译或反汇编本项目,除非适用法律明确允许此类活动。
## 5. 技术风险声明
⚠️ **使用本项目存在以下技术风险,用户应充分了解并自行承担:**
- 本工具需要访问聊天软件的数据库文件,可能因聊天软件版本更新导致功能失效或数据不兼容。
- 在 macOS 系统上使用时,需要临时关闭 SIP 安全机制,这可能降低系统安全性,用户应了解相关风险并自行决定是否使用。
- 本项目可能存在未知的技术缺陷或安全漏洞,可能导致数据损坏、丢失或泄露。
- 使用本项目处理大量数据可能导致系统性能下降或资源占用过高。
- 第三方依赖库或 API 的变更可能影响本项目的功能或安全性。
## 6. 禁止非法用途
严禁将本项目用于以下用途:
- 从事任何形式的非法活动,包括但不限于未授权系统测试、网络渗透或其他违反法律法规的行为。
- 监控、窃取或未经授权获取他人聊天记录或个人信息。
- 将获取的数据用于骚扰、诈骗、敲诈、威胁或其他侵害他人合法权益的行为。
- 规避任何安全措施或访问控制机制。
- 传播虚假信息、仇恨言论或违反公序良俗的内容。
- 侵犯任何第三方的知识产权、隐私权或其他合法权益。
**违反上述规定的,用户应自行承担全部法律责任,并赔偿因此给开发者或第三方造成的全部损失。**
## 7. 第三方服务集成
- 用户将聊天数据与第三方 LLM 服务(如 OpenAI、Claude 等)结合使用时,应仔细阅读并遵守这些服务的使用条款、隐私政策和数据处理协议。
- 用户应了解,向第三方服务传输数据可能导致数据离开用户控制范围,并受第三方服务条款约束。
- 本项目开发者不对第三方服务的可用性、安全性、准确性或数据处理行为负责,用户应自行评估相关风险。
- 用户应确保其向第三方服务传输数据的行为符合适用的数据保护法规和第三方服务条款。
## 8. 责任限制
**在法律允许的最大范围内:**
- 本项目按"原样"和"可用"状态提供,不对功能的适用性、可靠性、准确性、完整性或及时性做任何明示或暗示的保证。
- 开发者明确否认对适销性、特定用途适用性、不侵权以及任何其他明示或暗示的保证。
- 本项目开发者和贡献者不对用户使用本工具的行为及后果承担任何法律责任。
- 对于因使用本工具而可能导致的任何直接、间接、附带、特殊、惩罚性或后果性损失,包括但不限于数据丢失、业务中断、隐私泄露、声誉损害、利润损失、法律纠纷等,本项目开发者概不负责,即使开发者已被告知此类损失的可能性。
- 在任何情况下,开发者对用户的全部责任累计不超过用户为获取本软件实际支付的金额(如为免费获取则为零)。
## 9. 知识产权声明
- 本项目基于 Apache-2.0 许可证开源,用户在使用、修改和分发时应严格遵守该许可证的所有条款。
- 本项目的名称"Chatlog"、相关标识及商标权(如有)归开发者所有,未经明确授权,用户不得以任何方式使用这些标识进行商业活动。
- 根据 Apache-2.0 许可证,用户可自由使用、修改和分发本项目代码,但须遵守许可证规定的归属声明等要求。
- 用户对其修改版本自行承担全部责任,且不得以原项目名义发布,必须明确标明其为修改版本并与原项目区分。
- 用户不得移除或更改本项目中的版权声明、商标或其他所有权声明。
## 10. 数据处理合规性
- 用户在使用本项目处理个人数据时,应遵守适用的数据保护法规,包括但不限于《中华人民共和国个人信息保护法》、《通用数据保护条例》(GDPR)等。
- 用户应确保其具有处理相关数据的合法依据,如获得数据主体的明确同意。
- 用户应实施适当的技术和组织措施,确保数据安全,防止未授权访问、意外丢失或泄露。
- 在跨境传输数据时,用户应确保符合相关法律对数据出境的要求。
- 用户应尊重数据主体权利,包括访问权、更正权、删除权等。
## 11. 免责声明接受
下载、安装、使用本项目,表示用户已阅读、理解并同意遵守本免责声明的所有条款。如不同意,请立即停止使用本工具并删除相关代码和程序。
**用户确认:**
- 已完整阅读并理解本免责声明的全部内容
- 自愿接受本免责声明的全部条款
- 具有完全民事行为能力,能够理解并承担使用本项目的风险和责任
- 将遵守本免责声明中规定的所有义务和限制
## 12. 免责声明修改与通知
- 本免责声明可能根据项目发展和法律法规变化进行修改和调整,修改后的声明将在项目官方仓库页面公布。
- 开发者没有义务个别通知用户免责声明的变更,用户应定期查阅最新版本。
- 重大变更将通过项目仓库的 Release Notes 或 README 文件更新进行通知。
- 在免责声明更新后继续使用本项目,即视为接受修改后的条款。
## 13. 法律适用与管辖
- 本免责声明受中华人民共和国法律管辖,并按其解释。
- 任何与本免责声明有关的争议,应首先通过友好协商解决;协商不成的,提交至本项目开发者所在地有管辖权的人民法院诉讼解决。
- 对于中国境外用户,如本免责声明与用户所在地强制性法律规定冲突,应以不违反该强制性规定的方式解释和适用本声明,但本声明的其余部分仍然有效。
## 14. 可分割性
如本免责声明中的任何条款被有管辖权的法院或其他权威机构认定为无效、不合法或不可执行,不影响其余条款的有效性和可执行性。无效条款应被视为从本声明中分割,并在法律允许的最大范围内由最接近原条款意图的有效条款替代。
## 15. 完整协议
本免责声明构成用户与开发者之间关于本项目使用的完整协议,取代先前或同时期关于本项目的所有口头或书面协议、提议和陈述。本声明的任何豁免、修改或补充均应以书面形式作出并经开发者签署方为有效。

226
README.md
View File

@@ -2,7 +2,7 @@
# Chatlog
![chatlog](https://socialify.git.ci/sjzar/chatlog/image?font=Rokkitt&name=1&pattern=Diagonal+Stripes&theme=Auto)
![chatlog](https://socialify.git.ci/sjzar/chatlog/image?font=Rokkitt&forks=1&issues=1&name=1&pattern=Diagonal+Stripes&stargazers=1&theme=Auto)
_聊天记录工具,帮助大家轻松使用自己的聊天数据_
@@ -13,7 +13,7 @@ _聊天记录工具帮助大家轻松使用自己的聊天数据_
</div>
![chatlog](https://github.com/user-attachments/assets/746717b8-9b39-4a45-97f3-f0ae8fc5a344)
![chatlog](https://github.com/user-attachments/assets/e085d3a2-e009-4463-b2fd-8bd7df2b50c3)
## Feature
@@ -23,16 +23,35 @@ _聊天记录工具帮助大家轻松使用自己的聊天数据_
- 提供 Terminal UI 界面 & 命令行工具
- 提供 HTTP API 服务,支持查询聊天记录、联系人、群聊、最近会话等信息
- 支持 MCP SSE 协议,可与支持 MCP 的 AI 助手无缝集成
- 支持多媒体消息,支持解密图片、语音
- 支持自动解密数据,简化使用流程
- 支持多账号管理,可在不同账号间切换
## TODO
- 支持多媒体数据
- 聊天数据全文索引
- 聊天数据统计 & Dashboard
## Quick Start
## Install
### 基本步骤
1. **安装 Chatlog**[下载预编译版本](#下载预编译版本) 或 [使用 Go 安装](#从源码安装)
2. **运行程序**:执行 `chatlog` 启动 Terminal UI 界面
3. **解密数据**:选择 `解密数据` 菜单项
4. **开启 HTTP 服务**:选择 `开启 HTTP 服务` 菜单项
5. **访问数据**:通过 [HTTP API](#http-api) 或 [MCP 集成](#mcp-集成) 访问聊天记录
> 💡 **提示**:如果电脑端微信聊天记录不全,可以[从手机端迁移数据](#从手机迁移聊天记录)
### 常见问题快速解决
- **macOS 用户**:获取密钥前需[临时关闭 SIP](#macos-版本说明)
- **Windows 用户**:遇到界面显示问题请[使用 Windows Terminal](#windows-版本说明)
- **集成 AI 助手**:查看 [MCP 集成指南](#mcp-集成)
## 安装指南
### 从源码安装
@@ -44,143 +63,164 @@ go install github.com/sjzar/chatlog@latest
访问 [Releases](https://github.com/sjzar/chatlog/releases) 页面下载适合您系统的预编译版本。
## Quick Start
### 操作流程
1. 下载并安装微信客户端
2. 手机微信上操作 `我 - 设置 - 通用 - 聊天记录迁移与备份 - 迁移 - 迁移到电脑`,这一步的目的是将手机中的聊天记录传输到电脑上。可以放心操作,不会影响到手机上的聊天记录。
3. 下载 `chatlog` 预编译版本或从源码安装,推荐使用 go 进行安装。
4. 运行 `chatlog`,按照提示进行操作,解密数据并开启 HTTP 服务后,即可通过浏览器或 AI 助手访问聊天记录。
### macOS 版本提示
1. macOS 用户在获取密钥前,需要确认已经关闭 SIP 并安装 Xcode。由于 macOS 的安全机制,在正常情况在无法读取微信进程的内存数据,所以需要临时关闭 SIP。关闭 SIP 的方法:
```shell
# 1. 进入恢复模式
Apple Intel Mac: 关机后,按住 Command + R 键开机,直到出现苹果标志和进度条。
Apple Silicon Mac: 关机后,按住开机键不松开,直到出现苹果标志和进度条。
# 2. 打开终端
选项 - 实用工具 - 终端
# 3. 关闭 SIP
输入以下命令关闭 SIP
csrutil disable
# 4. 重启系统
```
2. 目前的 macOS 版本方案依赖 `lldb` 工具,所以需要安装 Xcode可以从 App Store 进行下载。
3. 仅获取数据密钥步骤需要关闭 SIP获取数据密钥后即可重新打开 SIP不影响解密数据和 HTTP 服务的运行。
## 使用指南
### Terminal UI 模式
1. 启动程序
最简单的使用方式是通过 Terminal UI 界面操作
```bash
./chatlog
chatlog
```
2. 使用界面操作:
- 使用方向键导航菜单
-`Enter` 选择菜单项
-`Esc` 返回上级菜单
-`Ctrl+C` 退出程序
操作方法
- 使用 `↑` `↓` 键选择菜单
-`Enter` 确认选择
-`Esc` 返回上级菜单
-`Ctrl+C` 退出程序
### 命令行模式
获取微信数据密钥
对于熟悉命令行的用户,可以直接使用以下命令
```bash
./chatlog key
# 获取微信数据密钥
chatlog key
# 解密数据库文件
chatlog decrypt
# 启动 HTTP 服务
chatlog server
```
解密数据库文件:
### 从手机迁移聊天记录
```bash
./chatlog decrypt
```
如果电脑端微信聊天记录不全,可以从手机端迁移数据:
1. 打开手机微信,进入 `我 - 设置 - 通用 - 聊天记录迁移与备份`
2. 选择 `迁移 - 迁移到电脑`,按照提示操作
3. 完成迁移后,重新运行 `chatlog` 获取密钥并解密数据
> 此操作不会影响手机上的聊天记录,只是将数据复制到电脑端
## 平台特定说明
### Windows 版本说明
如遇到界面显示异常(如花屏、乱码等),请使用 [Windows Terminal](https://github.com/microsoft/terminal) 运行程序
### macOS 版本说明
macOS 用户在获取密钥前需要临时关闭 SIP系统完整性保护
1. **关闭 SIP**
```shell
# 进入恢复模式
# Intel Mac: 重启时按住 Command + R
# Apple Silicon: 重启时长按电源键
# 在恢复模式中打开终端并执行
csrutil disable
# 重启系统
```
2. **安装必要工具**
```shell
# 安装 Xcode Command Line Tools
xcode-select --install
```
3. **获取密钥后**:可以重新启用 SIP`csrutil enable`),不影响后续使用
> Apple Silicon 用户注意确保微信、chatlog 和终端都不在 Rosetta 模式下运行
## HTTP API
启动 HTTP 服务后,可通过以下 API 访问数据:
启动 HTTP 服务后(默认地址 `http://127.0.0.1:5030`,可通过以下 API 访问数据:
### 聊天记录
### 聊天记录查询
```
GET /api/v1/chatlog?time=2023-01-01&talker=wxid_xxx&limit=100&offset=0&format=json
GET /api/v1/chatlog?time=2023-01-01&talker=wxid_xxx
```
参数说明:
- `time`: 时间范围,格式为 `YYYY-MM-DD` 或 `YYYY-MM-DD~YYYY-MM-DD`
- `talker`: 聊天对象的 ID不知道 ID 的话也可以尝试备注名、昵称、群聊 ID等
- `limit`: 返回记录数量限制
- `talker`: 聊天对象标识(支持 wxid、群聊 ID、备注名、昵称等
- `limit`: 返回记录数量
- `offset`: 分页偏移量
- `format`: 输出格式,支持 `json`、`csv` 或纯文本
### 联系人列表
### 其他 API 接口
```
GET /api/v1/contact
```
- **联系人列表**`GET /api/v1/contact`
- **群聊列表**`GET /api/v1/chatroom`
- **会话列表**`GET /api/v1/session`
- **多媒体内容**`GET /api/v1/media?msgid=xxx`
### 群聊列表
```
GET /api/v1/chatroom
```
## MCP 集成
### 会话列表
```
GET /api/v1/session
```
## MCP
支持 MCP SSE 协议,启动 HTTP 服务后,通过 SSE Endpoint 访问服务:
Chatlog 支持 MCP (Model Context Protocol) SSE 协议,可与支持 MCP 的 AI 助手无缝集成。
启动 HTTP 服务后,通过 SSE Endpoint 访问服务:
```
GET /sse
```
提供了 4 个 tool 用于与 AI 助手集成:
- `chatlog`: 查询聊天记录
- `query_contact`: 查询联系人
- `query_chat_room`: 查询群聊
- `query_recent_chat`: 查询最近会话
### 快速集成
### 示例
Chatlog 可以与多种支持 MCP 的 AI 助手集成,包括:
以 [ChatWise](https://chatwise.app/) 工具为例,在 `设置 - 工具` 下新建工具,类型为 `sse`ID 为 `chatlog`URL 为 `http://127.0.0.1:5030/sse`,勾选自动执行工具,即可使用。
- **ChatWise**: 直接支持 SSE在工具设置中添加 `http://127.0.0.1:5030/sse`
- **Cherry Studio**: 直接支持 SSE在 MCP 服务器设置中添加 `http://127.0.0.1:5030/sse`
部分 AI 聊天工具暂时不支持 MCP SSE 协议,可以通过 [`mcp-proxy`](https://github.com/sparfenyuk/mcp-proxy) 工具转发请求,以 [Claude Desktop](https://claude.ai/download) 为例,在安装好 `mcp-proxy` 后,将 `mcp-proxy` 配置到 `Claude Desktop``config.json` 文件中,即可使用
对于不直接支持 SSE 的客户端,可以使用 [mcp-proxy](https://github.com/sparfenyuk/mcp-proxy) 工具转发请求:
```json
{
"mcpServers": {
"mcp-proxy": {
"command": "/Users/sarv/.local/bin/mcp-proxy",
"args": [
"http://localhost:5030/sse"
],
"env": {}
}
},
"globalShortcut": ""
}
```
- **Claude Desktop**: 通过 mcp-proxy 支持,需要配置 `claude_desktop_config.json`
- **Monica Code**: 通过 mcp-proxy 支持,需要配置 VSCode 插件设置
### 详细集成指南
查看 [MCP 集成指南](docs/mcp.md) 获取各平台的详细配置步骤和注意事项。
## Prompt 示例
为了帮助大家更好地利用 Chatlog 与 AI 助手,我们整理了一些 prompt 示例。希望这些 prompt 可以启发大家更有效地查询和分析聊天记录,获取更精准的信息。
查看 [Prompt 指南](docs/prompt.md) 获取详细示例。
同时欢迎大家分享使用经验和 prompt如果您有好的 prompt 示例或使用技巧,请通过 [Discussions](https://github.com/sjzar/chatlog/discussions) 进行分享,共同进步。
## 免责声明
⚠️ **重要提示:使用本项目前,请务必阅读并理解完整的 [免责声明](./DISCLAIMER.md)。**
本项目仅供学习、研究和个人合法使用,禁止用于任何非法目的或未授权访问他人数据。下载、安装或使用本工具即表示您同意遵守免责声明中的所有条款,并自行承担使用过程中的全部风险和法律责任。
### 摘要(请阅读完整免责声明)
- 仅限处理您自己合法拥有的聊天数据或已获授权的数据
- 严禁用于未经授权获取、查看或分析他人聊天记录
- 开发者不对使用本工具可能导致的任何损失承担责任
- 使用第三方 LLM 服务时,您应遵守这些服务的使用条款和隐私政策
**本项目完全免费开源,任何以本项目名义收费的行为均与本项目无关。**
## License
`chatlog` 是在 Apache-2.0 许可下的开源软件
本项目基于 [Apache-2.0 许可证](./LICENSE) 开源
## 隐私政策
本项目不收集任何用户数据。所有数据处理均在用户本地设备上进行。使用第三方服务时,请参阅相应服务的隐私政策。
## Thanks
- [@0xlane](https://github.com/0xlane) 的 [wechat-dump-rs](https://github.com/0xlane/wechat-dump-rs) 项目
- [@xaoyaoo](https://github.com/xaoyaoo) 的 [PyWxDump](https://github.com/xaoyaoo/PyWxDump) 项目
- [@git-jiadong](https://github.com/git-jiadong) 的 [go-lame](https://github.com/git-jiadong/go-lame) 和 [go-silk](https://github.com/git-jiadong/go-silk) 项目
- [Anthropic](https://www.anthropic.com/) 的 [MCP]((https://github.com/modelcontextprotocol) ) 协议
- 各个 Go 开源库的贡献者们

43
cmd/chatlog/cmd_server.go Normal file
View File

@@ -0,0 +1,43 @@
package chatlog
import (
"runtime"
"github.com/sjzar/chatlog/internal/chatlog"
"github.com/rs/zerolog/log"
"github.com/spf13/cobra"
)
func init() {
rootCmd.AddCommand(serverCmd)
serverCmd.Flags().StringVarP(&serverAddr, "addr", "a", "127.0.0.1:5030", "server address")
serverCmd.Flags().StringVarP(&serverDataDir, "data-dir", "d", "", "data dir")
serverCmd.Flags().StringVarP(&serverWorkDir, "work-dir", "w", "", "work dir")
serverCmd.Flags().StringVarP(&serverPlatform, "platform", "p", runtime.GOOS, "platform")
serverCmd.Flags().IntVarP(&serverVer, "version", "v", 3, "version")
}
var (
serverAddr string
serverDataDir string
serverWorkDir string
serverPlatform string
serverVer int
)
var serverCmd = &cobra.Command{
Use: "server",
Short: "Start HTTP server",
Run: func(cmd *cobra.Command, args []string) {
m, err := chatlog.New("")
if err != nil {
log.Err(err).Msg("failed to create chatlog instance")
return
}
if err := m.CommandHTTPServer(serverAddr, serverDataDir, serverWorkDir, serverPlatform, serverVer); err != nil {
log.Err(err).Msg("failed to start server")
return
}
},
}

151
docs/mcp.md Normal file
View File

@@ -0,0 +1,151 @@
# MCP 集成指南
## 目录
- [MCP 集成指南](#mcp-集成指南)
- [目录](#目录)
- [前期准备](#前期准备)
- [mcp-proxy](#mcp-proxy)
- [ChatWise](#chatwise)
- [Cherry Studio](#cherry-studio)
- [Claude Desktop](#claude-desktop)
- [Monica Code](#monica-code)
## 前期准备
运行 `chatlog`,完成数据解密并开启 HTTP 服务
### mcp-proxy
如果遇到不支持 `SSE` 的客户端,可以尝试使用 `mcp-proxy``stdio` 的请求转换为 `SSE`
项目地址https://github.com/sparfenyuk/mcp-proxy
安装方式:
```shell
# 使用 uv 工具安装,也可参考项目文档的其他安装方式
uv tool install mcp-proxy
# 查询 mcp-proxy 的路径,后续可直接使用该路径
which mcp-proxy
/Users/sarv/.local/bin/mcp-proxy
```
## ChatWise
- 官网https://chatwise.app/
- 使用方式MCP SSE
- 注意事项:使用 ChatWise 的 MCP 功能需要 Pro 权限
1.`设置 - 工具` 下新建 `SSE 请求` 工具
![chatwise-1](https://github.com/user-attachments/assets/87e40f39-9fbc-4ff1-954a-d95548cde4c2)
1. 在 URL 中填写 `http://127.0.0.1:5030/sse`,并勾选 `自动执行工具`,点击 `查看工具` 即可检查连接 `chatlog` 是否正常
![chatwise-2](https://github.com/user-attachments/assets/8f98ef18-8e6c-40e6-ae78-8cd13e411c36)
3. 返回主页,选择支持 MCP 调用的模型,打开 `chatlog` 工具选项
![chatwise-3](https://github.com/user-attachments/assets/ea2aa178-5439-492b-a92f-4f4fc08828e7)
4. 测试功能是否正常
![chatwise-4](https://github.com/user-attachments/assets/8f82cb53-8372-40ee-a299-c02d3399403a)
## Cherry Studio
- 官网https://cherry-ai.com/
- 使用方式MCP SSE
1.`设置 - MCP 服务器` 下点击 `添加服务器`,输入名称为 `chatlog`,选择类型为 `服务器发送事件(sse)`,填写 URL 为 `http://127.0.0.1:5030/sse`,点击 `保存`。(注意:点击保存前不要先点击左侧的开启按钮)
![cherry-1](https://github.com/user-attachments/assets/93fc8b0a-9d95-499e-ab6c-e22b0c96fd6a)
2. 选择支持 MCP 调用的模型,打开 `chatlog` 工具选项
![cherry-2](https://github.com/user-attachments/assets/4e5bf752-2eab-4e7c-b73b-1b759d4a5f29)
3. 测试功能是否正常
![cherry-3](https://github.com/user-attachments/assets/c58a019f-fd5f-4fa3-830a-e81a60f2aa6f)
## Claude Desktop
- 官网https://claude.ai/download
- 使用方式mcp-proxy
- 参考资料https://modelcontextprotocol.io/quickstart/user#2-add-the-filesystem-mcp-server
1. 请先参考 [mcp-proxy](#mcp-proxy) 安装 `mcp-proxy`
2. 进入 Claude Desktop `Settings - Developer`,点击 `Edit Config` 按钮,这样会创建一个 `claude_desktop_config.json` 配置文件,并引导你编辑该文件
3. 编辑 `claude_desktop_config.json` 文件,配置名称为 `chatlog`command 为 `mcp-proxy` 的路径args 为 `http://127.0.0.1:5030/sse`,如下所示:
```json
{
"mcpServers": {
"chatlog": {
"command": "/Users/sarv/.local/bin/mcp-proxy",
"args": [
"http://localhost:5030/sse"
]
}
},
"globalShortcut": ""
}
```
4. 保存 `claude_desktop_config.json` 文件,重启 Claude Desktop可以看到 `chatlog` 已经添加成功
![claude-1](https://github.com/user-attachments/assets/f4e872cc-e6c1-4e24-97da-266466949cdf)
5. 测试功能是否正常
![claude-2](https://github.com/user-attachments/assets/832bb4d2-3639-4cbc-8b17-f4b812ea3637)
## Monica Code
- 官网https://monica.im/en/code
- 使用方式mcp-proxy
- 参考资料https://github.com/Monica-IM/Monica-Code/blob/main/Reference/config.md#modelcontextprotocolserver
1. 请先参考 [mcp-proxy](#mcp-proxy) 安装 `mcp-proxy`
2. 在 vscode 插件文件夹(`~/.vscode/extensions`)下找到 Monica Code 的目录,编辑 `config_schema.json` 文件。将 `experimental - modelContextProtocolServer``transport` 设置为如下内容:
```json
{
"experimental": {
"type": "object",
"title": "Experimental",
"description": "Experimental properties are subject to change.",
"properties": {
"modelContextProtocolServer": {
"type": "object",
"properties": {
"transport": {
"type": "stdio",
"command": "/Users/sarv/.local/bin/mcp-proxy",
"args": [
"http://localhost:5030/sse"
]
}
},
"required": [
"transport"
]
}
}
}
}
```
3. 重启 vscode可以看到 `chatlog` 已经添加成功
![monica-1](https://github.com/user-attachments/assets/8d0a96f2-ed05-48aa-a99a-06648ae1c500)
4. 测试功能是否正常
![monica-2](https://github.com/user-attachments/assets/054e0a30-428a-48a6-9f31-d2596fb8f743)

70
docs/prompt.md Normal file
View File

@@ -0,0 +1,70 @@
# Prompt 指南
## 概述
优秀的 `prompt` 可以极大的提高 `chatlog` 使用体验,收集了部分群友分享的 `prompt`,供大家参考。
在处理聊天记录时,尽量选择上下文长度足够的 LLM例如 `Gemini 2.5 Pro``Claude 3.5 Sonnet` 等。
欢迎大家在 [Discussions](https://github.com/sjzar/chatlog/discussions/47) 中分享自己的使用方式,共同进步。
## 群聊总结
作者:@eyaeya
```md
你是一个中文的群聊总结的助手,你可以为一个微信的群聊记录,提取并总结每个时间段大家在重点讨论的话题内容。
请帮我将 "<talker>" 在 <Time> 的群聊内容总结成一个群聊报告包含不多于5个的话题的总结如果还有更多话题可以在后面简单补充。每个话题包含以下内容
- 话题名(50字以内带序号1⃣2⃣3同时附带热度以🔥数量表示
- 参与者(不超过5个人将重复的人名去重)
- 时间段(从几点到几点)
- 过程(50到200字左右
- 评价(50字以下)
- 分割线: ------------
另外有以下要求:
1. 每个话题结束使用 ------------ 分割
2. 使用中文冒号
3. 无需大标题
4. 开始给出本群讨论风格的整体评价,例如活跃、太水、太黄、太暴力、话题不集中、无聊诸如此类
最后总结下最活跃的前五个发言者。
```
## 微信聊天记录可视化
作者:@数字声明卡兹克
原文地址https://mp.weixin.qq.com/s/Z66YRjY1EnC_hMgXE9_nnw
Prompt[微信聊天记录可视化prompt.txt](https://github.com/user-attachments/files/19773263/prompt.txt)
这份 prompt 可以使用聊天记录生成 HTML 网页,再使用 [YOURWARE](https://www.yourware.so/) 部署为可分享的静态网页。
### 技术讨论分析
作者:@eyaeya
```md
你作为一个专业的技术讨论分析者,请对以下聊天记录进行分析和结构化总结:
1. 基础信息提取:
- 将每个主题分成独立的问答对
- 保持原始对话的时间顺序
1. 问题分析要点:
- 提取问题的具体场景和背景
- 识别问题的核心技术难点
- 突出问题的实际影响
1. 解决方案总结:
- 列出具体的解决步骤
- 提取关键工具和资源
- 包含实践经验和注意事项
- 保留重要的链接和参考资料
1. 输出格式:
- 不要输出"日期:YYYY-MM-DD"这一行直接从问题1开始
- 问题1<简明扼要的问题描述>
- 回答1<完整的解决方案>
- 补充:<额外的讨论要点或注意事项>
1. 额外要求(严格执行)
- 如果有多个相关问题,保持逻辑顺序
- 标记重要的警告和建议、突出经验性的分享内容、保留有价值的专业术语解释、移除"我来分析"等过渡语确保链接的完整性
- 直接以日期开始,不要添加任何开场白
```

30
go.mod
View File

@@ -7,16 +7,18 @@ require (
github.com/gin-gonic/gin v1.10.0
github.com/google/uuid v1.6.0
github.com/klauspost/compress v1.18.0
github.com/mattn/go-sqlite3 v1.14.24
github.com/mattn/go-sqlite3 v1.14.27
github.com/pierrec/lz4/v4 v4.1.22
github.com/rivo/tview v0.0.0-20250325173046-7b72abf45814
github.com/rivo/tview v0.0.0-20250330220935-949945f8d922
github.com/rs/zerolog v1.34.0
github.com/shirou/gopsutil/v4 v4.25.2
github.com/shirou/gopsutil/v4 v4.25.3
github.com/sirupsen/logrus v1.9.3
github.com/sjzar/go-lame v0.0.8
github.com/sjzar/go-silk v0.0.1
github.com/spf13/cobra v1.9.1
github.com/spf13/viper v1.20.1
golang.org/x/crypto v0.36.0
golang.org/x/sys v0.31.0
golang.org/x/crypto v0.37.0
golang.org/x/sys v0.32.0
google.golang.org/protobuf v1.36.6
howett.net/plist v1.0.1
)
@@ -26,14 +28,14 @@ require (
github.com/bytedance/sonic/loader v0.2.4 // indirect
github.com/cloudwego/base64x v0.1.5 // indirect
github.com/ebitengine/purego v0.8.2 // indirect
github.com/fsnotify/fsnotify v1.8.0 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
github.com/gdamore/encoding v1.0.1 // indirect
github.com/gin-contrib/sse v1.0.0 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.25.0 // indirect
github.com/go-playground/validator/v10 v10.26.0 // indirect
github.com/go-viper/mapstructure/v2 v2.2.1 // indirect
github.com/goccy/go-json v0.10.5 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
@@ -42,12 +44,12 @@ require (
github.com/leodido/go-urn v1.4.0 // indirect
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-colorable v0.1.14 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.16 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.2.3 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/sagikazarmark/locafero v0.9.0 // indirect
@@ -62,9 +64,9 @@ require (
github.com/ugorji/go/codec v1.2.12 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/arch v0.15.0 // indirect
golang.org/x/net v0.38.0 // indirect
golang.org/x/term v0.30.0 // indirect
golang.org/x/text v0.23.0 // indirect
golang.org/x/arch v0.16.0 // indirect
golang.org/x/net v0.39.0 // indirect
golang.org/x/term v0.31.0 // indirect
golang.org/x/text v0.24.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)

61
go.sum
View File

@@ -15,16 +15,16 @@ github.com/ebitengine/purego v0.8.2 h1:jPPGWs2sZ1UgOSgD2bClL0MJIqu58nOmIcBuXr62z
github.com/ebitengine/purego v0.8.2/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.8.0 h1:dAwr6QBTBZIkG8roQaJjGof0pp0EeF+tNV7YBP3F/8M=
github.com/fsnotify/fsnotify v1.8.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/gabriel-vasile/mimetype v1.4.8 h1:FfZ3gj38NjllZIeJAmMhr+qKL8Wu+nOoI3GqacKw1NM=
github.com/gabriel-vasile/mimetype v1.4.8/go.mod h1:ByKUIKGjh1ODkGM1asKUbQZOLGrPjydw3hYPU2YU9t8=
github.com/gdamore/encoding v1.0.1 h1:YzKZckdBL6jVt2Gc+5p82qhrGiqMdG/eNs6Wy0u3Uhw=
github.com/gdamore/encoding v1.0.1/go.mod h1:0Z0cMFinngz9kS1QfMjCP8TY7em3bZYeeklsSDPivEo=
github.com/gdamore/tcell/v2 v2.8.1 h1:KPNxyqclpWpWQlPLx6Xui1pMk8S+7+R37h3g07997NU=
github.com/gdamore/tcell/v2 v2.8.1/go.mod h1:bj8ori1BG3OYMjmb3IklZVWfZUJ1UBQt9JXrOCOhGWw=
github.com/gin-contrib/sse v1.0.0 h1:y3bT1mUWUxDpW4JLQg/HnTqV4rozuW4tC9eFKTxYI9E=
github.com/gin-contrib/sse v1.0.0/go.mod h1:zNuFdwarAygJBht0NTKiSi3jRf6RbqeILZ9Sp6Slhe0=
github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
github.com/gin-gonic/gin v1.10.0 h1:nTuyha1TYqgedzytsKYqna+DfLos46nTv2ygFy86HFU=
github.com/gin-gonic/gin v1.10.0/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
@@ -36,8 +36,8 @@ github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/o
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.25.0 h1:5Dh7cjvzR7BRZadnsVOzPhWsrwUr0nmsZJxEAnFLNO8=
github.com/go-playground/validator/v10 v10.25.0/go.mod h1:GGzBIJMuE98Ic/kJsBXbz1x/7cByt++cQ+YOuDM5wus=
github.com/go-playground/validator/v10 v10.26.0 h1:SP05Nqhjcvz81uJaRfEV0YBSSSGMc/iMaVtFbr3Sw2k=
github.com/go-playground/validator/v10 v10.26.0/go.mod h1:I5QpIEbmr8On7W0TktmJAumgzX4CA1XNl4ZmDuVHKKo=
github.com/go-viper/mapstructure/v2 v2.2.1 h1:ZAaOCxANMuZx5RCeg0mBdEZk7DZasvvZIxtHqx8aGss=
github.com/go-viper/mapstructure/v2 v2.2.1/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
@@ -70,23 +70,24 @@ github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69
github.com/lucasb-eyer/go-colorful v1.2.0/go.mod h1:R4dSotOR9KMtayYi1e77YzuveK+i7ruzyGqttikkLy0=
github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35 h1:PpXWgLPs+Fqr325bN2FD2ISlRRztXibcX6e8f5FR5Dc=
github.com/lufia/plan9stats v0.0.0-20250317134145-8bc96cf8fc35/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg=
github.com/mattn/go-colorable v0.1.13 h1:fFA4WZxdEF4tXPZVKMLwD8oUnCTTo08duU7wxecdEvA=
github.com/mattn/go-colorable v0.1.13/go.mod h1:7S9/ev0klgBDR4GtXTXX8a3vIGJpMovkB8vQcUbaXHg=
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
github.com/mattn/go-colorable v0.1.14/go.mod h1:6LmQG8QLFO4G5z1gPvYEzlUgJ2wF+stgPZH1UqBm1s8=
github.com/mattn/go-isatty v0.0.16/go.mod h1:kYGgaQfpe5nmfYZH+SKPsOc2e4SrIfOl2e/yFXSvRLM=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/mattn/go-runewidth v0.0.16 h1:E5ScNMtiwvlvB5paMFdw9p4kSQzbXFikJ5SQO6TULQc=
github.com/mattn/go-runewidth v0.0.16/go.mod h1:Jdepj2loyihRzMpdS35Xk/zdY8IAYHsh153qUoGf23w=
github.com/mattn/go-sqlite3 v1.14.24 h1:tpSp2G2KyMnnQu99ngJ47EIkWVmliIizyZBfPrBWDRM=
github.com/mattn/go-sqlite3 v1.14.24/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/mattn/go-sqlite3 v1.14.27 h1:drZCnuvf37yPfs95E5jd9s3XhdVWLal+6BOK6qrv6IU=
github.com/mattn/go-sqlite3 v1.14.27/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/pelletier/go-toml/v2 v2.2.3 h1:YmeHyLY8mFWbdkNWwpr+qIL2bEqT0o95WSdkNHvL12M=
github.com/pelletier/go-toml/v2 v2.2.3/go.mod h1:MfCQTFTvCcUyyvvwm1+G6H/jORL20Xlb6rzQu9GuUkc=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pierrec/lz4/v4 v4.1.22 h1:cKFw6uJDK+/gfw5BcDL0JL5aBsAFdsIT18eRtLj7VIU=
github.com/pierrec/lz4/v4 v4.1.22/go.mod h1:gZWDp/Ze/IJXGXf23ltt2EXimqmTUXEy0GFuRQyBid4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -94,8 +95,8 @@ github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZb
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/rivo/tview v0.0.0-20250325173046-7b72abf45814 h1:pJIO3sp+rkDbJTeqqpe2Oihq3hegiM5ASvsd6S0pvjg=
github.com/rivo/tview v0.0.0-20250325173046-7b72abf45814/go.mod h1:02iFIz7K/A9jGCvrizLPvoqr4cEIx7q54RH5Qudkrss=
github.com/rivo/tview v0.0.0-20250330220935-949945f8d922 h1:SMyqkaRfpE8ZQUSRTZKO3uN84xov++OGa+e3NCksaQw=
github.com/rivo/tview v0.0.0-20250330220935-949945f8d922/go.mod h1:02iFIz7K/A9jGCvrizLPvoqr4cEIx7q54RH5Qudkrss=
github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
github.com/rivo/uniseg v0.4.3/go.mod h1:FN3SvrM+Zdj16jyLfmOkMNblXMcoc8DfTHruCPUcx88=
github.com/rivo/uniseg v0.4.7 h1:WUdvkW8uEhrYfLC4ZzdpI2ztxP1I582+49Oc5Mq64VQ=
@@ -108,10 +109,14 @@ github.com/rs/zerolog v1.34.0/go.mod h1:bJsvje4Z08ROH4Nhs5iH600c3IkWhwp44iRc54W6
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sagikazarmark/locafero v0.9.0 h1:GbgQGNtTrEmddYDSAH9QLRyfAHY12md+8YFTqyMTC9k=
github.com/sagikazarmark/locafero v0.9.0/go.mod h1:UBUyz37V+EdMS3hDF3QWIiVr/2dPrx49OMO0Bn0hJqk=
github.com/shirou/gopsutil/v4 v4.25.2 h1:NMscG3l2CqtWFS86kj3vP7soOczqrQYIEhO/pMvvQkk=
github.com/shirou/gopsutil/v4 v4.25.2/go.mod h1:34gBYJzyqCDT11b6bMHP0XCvWeU3J61XRT7a2EmCRTA=
github.com/shirou/gopsutil/v4 v4.25.3 h1:SeA68lsu8gLggyMbmCn8cmp97V1TI9ld9sVzAUcKcKE=
github.com/shirou/gopsutil/v4 v4.25.3/go.mod h1:xbuxyoZj+UsgnZrENu3lQivsngRR5BdjbJwf2fv4szA=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/sjzar/go-lame v0.0.8 h1:AS9l32R6foMiMEXWfUY8i79WIMfDoBC2QqQ9s5yziIk=
github.com/sjzar/go-lame v0.0.8/go.mod h1:8RmqWcAKSbBAk6bTRV9d8mdDxqK3hY9vFyoJ4DoQE6Y=
github.com/sjzar/go-silk v0.0.1 h1:cXD9dsIZti3n+g0Fd3IUvLH9A7tyL4jvUsHEyhff21s=
github.com/sjzar/go-silk v0.0.1/go.mod h1:IXVcHEXKiU9j3ZtHEiGS37OFKkex9pdAhZVcFzAIOlM=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
github.com/sourcegraph/conc v0.3.0/go.mod h1:Sdozi7LEKbFPqYX2/J+iBAM6HpqSLTASQIKqDmF7Mt0=
github.com/spf13/afero v1.14.0 h1:9tH6MapGnn/j0eb0yIXiLjERO8RB6xIVZRDCX7PtqWA=
@@ -127,13 +132,11 @@ github.com/spf13/viper v1.20.1/go.mod h1:P9Mdzt1zoHIG8m2eZQinpiBjo6kCmZSKBClNNqj
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA=
github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
@@ -151,15 +154,15 @@ github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
golang.org/x/arch v0.15.0 h1:QtOrQd0bTUnhNVNndMpLHNWrDmYzZ2KDqSrEymqInZw=
golang.org/x/arch v0.15.0/go.mod h1:JmwW7aLIoRUKgaTzhkiEFxvcEiQGyOg9BMonBJUS7EE=
golang.org/x/arch v0.16.0 h1:foMtLTdyOmIniqWCHjY6+JxuC54XP1fDwx4N0ASyW+U=
golang.org/x/arch v0.16.0/go.mod h1:JmwW7aLIoRUKgaTzhkiEFxvcEiQGyOg9BMonBJUS7EE=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
golang.org/x/crypto v0.19.0/go.mod h1:Iy9bg/ha4yyC70EfRS8jz+B6ybOBKMaSxLj6P6oBDfU=
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
golang.org/x/crypto v0.36.0 h1:AnAEvhDddvBdpY+uR+MyHmuZzzNqXSe/GvuDeob5L34=
golang.org/x/crypto v0.36.0/go.mod h1:Y4J0ReaxCR1IMaabaSMugxJES1EpwhBHhv2bDHklZvc=
golang.org/x/crypto v0.37.0 h1:kJNSjF/Xp7kU0iB2Z+9viTPMW4EqqsrywMXLJOOsXSE=
golang.org/x/crypto v0.37.0/go.mod h1:vg+k43peMZ0pUMhYmVAWysMK35e6ioLh3wB8ZCAfbVc=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
golang.org/x/mod v0.8.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
golang.org/x/mod v0.12.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs=
@@ -173,8 +176,8 @@ golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
golang.org/x/net v0.15.0/go.mod h1:idbUs1IY1+zTqbi8yxTbhexhEEk5ur9LInksu6HrEpk=
golang.org/x/net v0.21.0/go.mod h1:bIjVDfnllIU7BJ2DNgfnXvpSvtn8VRwhlsaeUTyUS44=
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
golang.org/x/net v0.38.0 h1:vRMAPTMaeGqVhG5QyLJHqNDwecKTomGeqbnfZyKlBI8=
golang.org/x/net v0.38.0/go.mod h1:ivrbrMbzFq5J41QOQh0siUuly180yBYtLp+CKbEaFx8=
golang.org/x/net v0.39.0 h1:ZCu7HMWDxpXpaiKdhzIfaltL9Lp31x/3fCP11bc6/fY=
golang.org/x/net v0.39.0/go.mod h1:X7NRbYVEA+ewNkCNyJ513WmMdQ3BineSwVtN2zD/d+E=
golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -199,8 +202,8 @@ golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.17.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.29.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.31.0 h1:ioabZlmFYtWhL+TRYpcnNlLwhyxaM9kWTDEmfnprqik=
golang.org/x/sys v0.31.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/sys v0.32.0 h1:s77OFDvIQeibCmezSnk/q6iAfkdiQaJi4VzroCFrN20=
golang.org/x/sys v0.32.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
golang.org/x/telemetry v0.0.0-20240228155512-f48c80bd79b2/go.mod h1:TeRTkGYfJXctD9OcfyVLyj2J3IxLnKwHJR8f4D8a3YE=
golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
@@ -210,8 +213,8 @@ golang.org/x/term v0.12.0/go.mod h1:owVbMEjm3cBLCHdkQu9b1opXd4ETQWc3BhuQGKgXgvU=
golang.org/x/term v0.17.0/go.mod h1:lLRBjIVuehSbZlaOtGMbcMncT+aqLLLmKrsjNrUguwk=
golang.org/x/term v0.20.0/go.mod h1:8UkIAJTvZgivsXaD6/pH6U9ecQzZ45awqEOzuCvwpFY=
golang.org/x/term v0.28.0/go.mod h1:Sw/lC2IAUZ92udQNf3WodGtn4k/XoLyZoh8v/8uiwek=
golang.org/x/term v0.30.0 h1:PQ39fJZ+mfadBm0y5WlL4vlM7Sx1Hgf13sMIY2+QS9Y=
golang.org/x/term v0.30.0/go.mod h1:NYYFdzHoI5wRh/h5tDMdMqCqPJZEuNqVR5xJLd/n67g=
golang.org/x/term v0.31.0 h1:erwDkOK1Msy6offm1mOgvspSkslFnIGsFnxOKoufg3o=
golang.org/x/term v0.31.0/go.mod h1:R4BeIy7D95HzImkxGkTW1UQTtP54tio2RyHz7PwK0aw=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ=
@@ -221,8 +224,8 @@ golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
golang.org/x/text v0.23.0 h1:D71I7dUrlY+VX0gQShAThNGHFxZ13dGLBHQLVl1mJlY=
golang.org/x/text v0.23.0/go.mod h1:/BLNzu4aZCJ1+kcD0DNRotWKage4q2rGVAg4o22unh4=
golang.org/x/text v0.24.0 h1:dd5Bzh4yt5KYA8f9CJHCP4FB4D51c2c6JvN37xJJkJ0=
golang.org/x/text v0.24.0/go.mod h1:L8rBsPeo2pSS+xqN0d5u2ikmjtmoJbDBT1b7nHvFCdU=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20191119224855-298f0cb1881e/go.mod h1:b+2E5dAYhXwXZwtnZ6UAqBI28+e2cm9otk0dWdXHAEo=
golang.org/x/tools v0.1.12/go.mod h1:hNGJHUnrk76NpqgfD5Aqm5Crs+Hm0VOH/i9J2+nxYbc=

View File

@@ -2,14 +2,17 @@ package chatlog
import (
"fmt"
"path/filepath"
"runtime"
"time"
"github.com/sjzar/chatlog/internal/chatlog/ctx"
"github.com/sjzar/chatlog/internal/ui/footer"
"github.com/sjzar/chatlog/internal/ui/form"
"github.com/sjzar/chatlog/internal/ui/help"
"github.com/sjzar/chatlog/internal/ui/infobar"
"github.com/sjzar/chatlog/internal/ui/menu"
"github.com/sjzar/chatlog/internal/wechat"
"github.com/gdamore/tcell/v2"
"github.com/rivo/tview"
@@ -54,6 +57,8 @@ func NewApp(ctx *ctx.Context, m *Manager) *App {
app.initMenu()
app.updateMenuItemsState()
return app
}
@@ -91,6 +96,33 @@ func (a *App) Stop() {
a.Application.Stop()
}
func (a *App) updateMenuItemsState() {
// 查找并更新自动解密菜单项
for _, item := range a.menu.GetItems() {
// 更新自动解密菜单项
if item.Index == 5 {
if a.ctx.AutoDecrypt {
item.Name = "停止自动解密"
item.Description = "停止监控数据目录更新,不再自动解密新增数据"
} else {
item.Name = "开启自动解密"
item.Description = "监控数据目录更新,自动解密新增数据"
}
}
// 更新HTTP服务菜单项
if item.Index == 4 {
if a.ctx.HTTPEnabled {
item.Name = "停止 HTTP 服务"
item.Description = "停止本地 HTTP & MCP 服务器"
} else {
item.Name = "启动 HTTP 服务"
item.Description = "启动本地 HTTP & MCP 服务器"
}
}
}
}
func (a *App) switchTab(step int) {
index := (a.activeTab + step) % a.tabCount
if index < 0 {
@@ -109,17 +141,29 @@ func (a *App) refresh() {
case <-a.stopRefresh:
return
case <-tick.C:
if a.ctx.AutoDecrypt || a.ctx.HTTPEnabled {
a.m.RefreshSession()
}
a.infoBar.UpdateAccount(a.ctx.Account)
a.infoBar.UpdateBasicInfo(a.ctx.PID, a.ctx.FullVersion, a.ctx.ExePath)
a.infoBar.UpdateStatus(a.ctx.Status)
a.infoBar.UpdateDataKey(a.ctx.DataKey)
a.infoBar.UpdatePlatform(a.ctx.Platform)
a.infoBar.UpdateDataUsageDir(a.ctx.DataUsage, a.ctx.DataDir)
a.infoBar.UpdateWorkUsageDir(a.ctx.WorkUsage, a.ctx.WorkDir)
if a.ctx.LastSession.Unix() > 1000000000 {
a.infoBar.UpdateSession(a.ctx.LastSession.Format("2006-01-02 15:04:05"))
}
if a.ctx.HTTPEnabled {
a.infoBar.UpdateHTTPServer(fmt.Sprintf("[green][已启动][white] [%s]", a.ctx.HTTPAddr))
} else {
a.infoBar.UpdateHTTPServer("[未启动]")
}
if a.ctx.AutoDecrypt {
a.infoBar.UpdateAutoDecrypt("[green][已开启][white]")
} else {
a.infoBar.UpdateAutoDecrypt("[未开启]")
}
a.Draw()
}
@@ -257,11 +301,11 @@ func (a *App) initMenu() {
} else {
// 启动成功
modal.SetText("已启动 HTTP 服务")
// 更改菜单项名称
i.Name = "停止 HTTP 服务"
i.Description = "停止本地 HTTP & MCP 服务器"
}
// 更改菜单项名称
a.updateMenuItemsState()
// 添加确认按钮
modal.AddButtons([]string{"OK"})
modal.SetDoneFunc(func(buttonIndex int, buttonLabel string) {
@@ -288,11 +332,89 @@ func (a *App) initMenu() {
} else {
// 停止成功
modal.SetText("已停止 HTTP 服务")
// 更改菜单项名称
i.Name = "启动 HTTP 服务"
i.Description = "启动本地 HTTP & MCP 服务器"
}
// 更改菜单项名称
a.updateMenuItemsState()
// 添加确认按钮
modal.AddButtons([]string{"OK"})
modal.SetDoneFunc(func(buttonIndex int, buttonLabel string) {
a.mainPages.RemovePage("modal")
})
a.SetFocus(modal)
})
}()
}
},
}
autoDecrypt := &menu.Item{
Index: 5,
Name: "开启自动解密",
Description: "自动解密新增的数据文件",
Selected: func(i *menu.Item) {
modal := tview.NewModal()
// 根据当前自动解密状态执行不同操作
if !a.ctx.AutoDecrypt {
// 自动解密未开启,开启自动解密
modal.SetText("正在开启自动解密...")
a.mainPages.AddPage("modal", modal, true, true)
a.SetFocus(modal)
// 在后台开启自动解密
go func() {
err := a.m.StartAutoDecrypt()
// 在主线程中更新UI
a.QueueUpdateDraw(func() {
if err != nil {
// 开启失败
modal.SetText("开启自动解密失败: " + err.Error())
} else {
// 开启成功
if a.ctx.Version == 3 {
modal.SetText("已开启自动解密\n3.x版本数据文件更新不及时有低延迟需求请使用4.0版本")
} else {
modal.SetText("已开启自动解密")
}
}
// 更改菜单项名称
a.updateMenuItemsState()
// 添加确认按钮
modal.AddButtons([]string{"OK"})
modal.SetDoneFunc(func(buttonIndex int, buttonLabel string) {
a.mainPages.RemovePage("modal")
})
a.SetFocus(modal)
})
}()
} else {
// 自动解密已开启,停止自动解密
modal.SetText("正在停止自动解密...")
a.mainPages.AddPage("modal", modal, true, true)
a.SetFocus(modal)
// 在后台停止自动解密
go func() {
err := a.m.StopAutoDecrypt()
// 在主线程中更新UI
a.QueueUpdateDraw(func() {
if err != nil {
// 停止失败
modal.SetText("停止自动解密失败: " + err.Error())
} else {
// 停止成功
modal.SetText("已停止自动解密")
}
// 更改菜单项名称
a.updateMenuItemsState()
// 添加确认按钮
modal.AddButtons([]string{"OK"})
modal.SetDoneFunc(func(buttonIndex int, buttonLabel string) {
@@ -306,19 +428,28 @@ func (a *App) initMenu() {
}
setting := &menu.Item{
Index: 5,
Index: 6,
Name: "设置",
Description: "设置应用程序选项",
Selected: a.settingSelected,
}
a.menu.AddItem(setting)
selectAccount := &menu.Item{
Index: 7,
Name: "切换账号",
Description: "切换当前操作的账号,可以选择进程或历史账号",
Selected: a.selectAccountSelected,
}
a.menu.AddItem(getDataKey)
a.menu.AddItem(decryptData)
a.menu.AddItem(httpServer)
a.menu.AddItem(autoDecrypt)
a.menu.AddItem(setting)
a.menu.AddItem(selectAccount)
a.menu.AddItem(&menu.Item{
Index: 6,
Index: 8,
Name: "退出",
Description: "退出程序",
Selected: func(i *menu.Item) {
@@ -347,6 +478,16 @@ func (a *App) settingSelected(i *menu.Item) {
description: "配置数据解密后的存储目录",
action: a.settingWorkDir,
},
{
name: "设置数据密钥",
description: "配置数据解密密钥",
action: a.settingDataKey,
},
{
name: "设置数据目录",
description: "配置微信数据文件所在目录",
action: a.settingDataDir,
},
}
subMenu := menu.NewSubMenu("设置")
@@ -370,43 +511,279 @@ func (a *App) settingSelected(i *menu.Item) {
// settingHTTPPort 设置 HTTP 端口
func (a *App) settingHTTPPort() {
// 实现端口设置逻辑
// 这里可以使用 tview.InputField 让用户输入端口
form := tview.NewForm().
AddInputField("地址", a.ctx.HTTPAddr, 20, nil, func(text string) {
a.m.SetHTTPAddr(text)
}).
AddButton("保存", func() {
// 使用我们的自定义表单组件
formView := form.NewForm("设置 HTTP 地址")
// 临时存储用户输入的值
tempHTTPAddr := a.ctx.HTTPAddr
// 添加输入字段 - 不再直接设置HTTP地址而是更新临时变量
formView.AddInputField("地址", tempHTTPAddr, 0, nil, func(text string) {
tempHTTPAddr = text // 只更新临时变量
})
// 添加按钮 - 点击保存时才设置HTTP地址
formView.AddButton("保存", func() {
a.m.SetHTTPAddr(tempHTTPAddr) // 在这里设置HTTP地址
a.mainPages.RemovePage("submenu2")
a.showInfo("HTTP 地址已设置为 " + a.ctx.HTTPAddr)
}).
AddButton("取消", func() {
})
formView.AddButton("取消", func() {
a.mainPages.RemovePage("submenu2")
})
form.SetBorder(true).SetTitle("设置 HTTP 地址")
a.mainPages.AddPage("submenu2", form, true, true)
a.SetFocus(form)
a.mainPages.AddPage("submenu2", formView, true, true)
a.SetFocus(formView)
}
// settingWorkDir 设置工作目录
func (a *App) settingWorkDir() {
// 实现工作目录设置逻辑
form := tview.NewForm().
AddInputField("工作目录", a.ctx.WorkDir, 40, nil, func(text string) {
a.ctx.SetWorkDir(text)
}).
AddButton("保存", func() {
// 使用我们的自定义表单组件
formView := form.NewForm("设置工作目录")
// 临时存储用户输入的值
tempWorkDir := a.ctx.WorkDir
// 添加输入字段 - 不再直接设置工作目录,而是更新临时变量
formView.AddInputField("工作目录", tempWorkDir, 0, nil, func(text string) {
tempWorkDir = text // 只更新临时变量
})
// 添加按钮 - 点击保存时才设置工作目录
formView.AddButton("保存", func() {
a.ctx.SetWorkDir(tempWorkDir) // 在这里设置工作目录
a.mainPages.RemovePage("submenu2")
a.showInfo("工作目录已设置为 " + a.ctx.WorkDir)
}).
AddButton("取消", func() {
})
formView.AddButton("取消", func() {
a.mainPages.RemovePage("submenu2")
})
form.SetBorder(true).SetTitle("设置工作目录")
a.mainPages.AddPage("submenu2", form, true, true)
a.SetFocus(form)
a.mainPages.AddPage("submenu2", formView, true, true)
a.SetFocus(formView)
}
// settingDataKey 设置数据密钥
func (a *App) settingDataKey() {
// 使用我们的自定义表单组件
formView := form.NewForm("设置数据密钥")
// 临时存储用户输入的值
tempDataKey := a.ctx.DataKey
// 添加输入字段 - 不直接设置数据密钥,而是更新临时变量
formView.AddInputField("数据密钥", tempDataKey, 0, nil, func(text string) {
tempDataKey = text // 只更新临时变量
})
// 添加按钮 - 点击保存时才设置数据密钥
formView.AddButton("保存", func() {
a.ctx.DataKey = tempDataKey // 设置数据密钥
a.mainPages.RemovePage("submenu2")
a.showInfo("数据密钥已设置")
})
formView.AddButton("取消", func() {
a.mainPages.RemovePage("submenu2")
})
a.mainPages.AddPage("submenu2", formView, true, true)
a.SetFocus(formView)
}
// settingDataDir 设置数据目录
func (a *App) settingDataDir() {
// 使用我们的自定义表单组件
formView := form.NewForm("设置数据目录")
// 临时存储用户输入的值
tempDataDir := a.ctx.DataDir
// 添加输入字段 - 不直接设置数据目录,而是更新临时变量
formView.AddInputField("数据目录", tempDataDir, 0, nil, func(text string) {
tempDataDir = text // 只更新临时变量
})
// 添加按钮 - 点击保存时才设置数据目录
formView.AddButton("保存", func() {
a.ctx.DataDir = tempDataDir // 设置数据目录
a.mainPages.RemovePage("submenu2")
a.showInfo("数据目录已设置为 " + a.ctx.DataDir)
})
formView.AddButton("取消", func() {
a.mainPages.RemovePage("submenu2")
})
a.mainPages.AddPage("submenu2", formView, true, true)
a.SetFocus(formView)
}
// selectAccountSelected 处理切换账号菜单项的选择事件
func (a *App) selectAccountSelected(i *menu.Item) {
// 创建子菜单
subMenu := menu.NewSubMenu("切换账号")
// 添加微信进程
instances := a.m.wechat.GetWeChatInstances()
if len(instances) > 0 {
// 添加实例标题
subMenu.AddItem(&menu.Item{
Index: 0,
Name: "--- 微信进程 ---",
Description: "",
Hidden: false,
Selected: nil,
})
// 添加实例列表
for idx, instance := range instances {
// 创建一个实例描述
description := fmt.Sprintf("版本: %s 目录: %s", instance.FullVersion, instance.DataDir)
// 标记当前选中的实例
name := fmt.Sprintf("%s [%d]", instance.Name, instance.PID)
if a.ctx.Current != nil && a.ctx.Current.PID == instance.PID {
name = name + " [当前]"
}
// 创建菜单项
instanceItem := &menu.Item{
Index: idx + 1,
Name: name,
Description: description,
Hidden: false,
Selected: func(instance *wechat.Account) func(*menu.Item) {
return func(*menu.Item) {
// 如果是当前账号,则无需切换
if a.ctx.Current != nil && a.ctx.Current.PID == instance.PID {
a.mainPages.RemovePage("submenu")
a.showInfo("已经是当前账号")
return
}
// 显示切换中的模态框
modal := tview.NewModal().SetText("正在切换账号...")
a.mainPages.AddPage("modal", modal, true, true)
a.SetFocus(modal)
// 在后台执行切换操作
go func() {
err := a.m.Switch(instance, "")
// 在主线程中更新UI
a.QueueUpdateDraw(func() {
a.mainPages.RemovePage("modal")
a.mainPages.RemovePage("submenu")
if err != nil {
// 切换失败
a.showError(fmt.Errorf("切换账号失败: %v", err))
} else {
// 切换成功
a.showInfo("切换账号成功")
// 更新菜单状态
a.updateMenuItemsState()
}
})
}()
}
}(instance),
}
subMenu.AddItem(instanceItem)
}
}
// 添加历史账号
if len(a.ctx.History) > 0 {
// 添加历史账号标题
subMenu.AddItem(&menu.Item{
Index: 100,
Name: "--- 历史账号 ---",
Description: "",
Hidden: false,
Selected: nil,
})
// 添加历史账号列表
idx := 101
for account, hist := range a.ctx.History {
// 创建一个账号描述
description := fmt.Sprintf("版本: %s 目录: %s", hist.FullVersion, hist.DataDir)
// 标记当前选中的账号
name := account
if name == "" {
name = filepath.Base(hist.DataDir)
}
if a.ctx.DataDir == hist.DataDir {
name = name + " [当前]"
}
// 创建菜单项
histItem := &menu.Item{
Index: idx,
Name: name,
Description: description,
Hidden: false,
Selected: func(account string) func(*menu.Item) {
return func(*menu.Item) {
// 如果是当前账号,则无需切换
if a.ctx.Current != nil && a.ctx.DataDir == a.ctx.History[account].DataDir {
a.mainPages.RemovePage("submenu")
a.showInfo("已经是当前账号")
return
}
// 显示切换中的模态框
modal := tview.NewModal().SetText("正在切换账号...")
a.mainPages.AddPage("modal", modal, true, true)
a.SetFocus(modal)
// 在后台执行切换操作
go func() {
err := a.m.Switch(nil, account)
// 在主线程中更新UI
a.QueueUpdateDraw(func() {
a.mainPages.RemovePage("modal")
a.mainPages.RemovePage("submenu")
if err != nil {
// 切换失败
a.showError(fmt.Errorf("切换账号失败: %v", err))
} else {
// 切换成功
a.showInfo("切换账号成功")
// 更新菜单状态
a.updateMenuItemsState()
}
})
}()
}
}(account),
}
idx++
subMenu.AddItem(histItem)
}
}
// 如果没有账号可选择
if len(a.ctx.History) == 0 && len(instances) == 0 {
subMenu.AddItem(&menu.Item{
Index: 1,
Name: "无可用账号",
Description: "未检测到微信进程或历史账号",
Hidden: false,
Selected: nil,
})
}
// 显示子菜单
a.mainPages.AddPage("submenu", subMenu, true, true)
a.SetFocus(subMenu)
}
// showModal 显示一个模态对话框

View File

@@ -2,6 +2,7 @@ package ctx
import (
"sync"
"time"
"github.com/sjzar/chatlog/internal/chatlog/conf"
"github.com/sjzar/chatlog/internal/wechat"
@@ -33,6 +34,10 @@ type Context struct {
HTTPEnabled bool
HTTPAddr string
// 自动解密
AutoDecrypt bool
LastSession time.Time
// 当前选中的微信实例
Current *wechat.Account
PID int
@@ -63,6 +68,10 @@ func (c *Context) loadConfig() {
func (c *Context) SwitchHistory(account string) {
c.mu.Lock()
defer c.mu.Unlock()
c.Current = nil
c.PID = 0
c.ExePath = ""
c.Status = ""
history, ok := c.History[account]
if ok {
c.Account = history.Account
@@ -153,6 +162,13 @@ func (c *Context) SetDataDir(dir string) {
c.Refresh()
}
func (c *Context) SetAutoDecrypt(enabled bool) {
c.mu.Lock()
defer c.mu.Unlock()
c.AutoDecrypt = enabled
c.UpdateConfig()
}
// 更新配置
func (c *Context) UpdateConfig() {
pconf := conf.ProcessConfig{

View File

@@ -40,8 +40,8 @@ func (s *Service) GetDB() *wechatdb.DB {
return s.db
}
func (s *Service) GetMessages(start, end time.Time, talker string, limit, offset int) ([]*model.Message, error) {
return s.db.GetMessages(start, end, talker, limit, offset)
func (s *Service) GetMessages(start, end time.Time, talker string, sender string, keyword string, limit, offset int) ([]*model.Message, error) {
return s.db.GetMessages(start, end, talker, sender, keyword, limit, offset)
}
func (s *Service) GetContacts(key string, limit, offset int) (*wechatdb.GetContactsResp, error) {

View File

@@ -12,6 +12,7 @@ import (
"github.com/sjzar/chatlog/internal/errors"
"github.com/sjzar/chatlog/pkg/util"
"github.com/sjzar/chatlog/pkg/util/dat2img"
"github.com/sjzar/chatlog/pkg/util/silk"
"github.com/gin-gonic/gin"
)
@@ -34,6 +35,7 @@ func (s *Service) initRouter() {
router.GET("/image/:key", s.GetImage)
router.GET("/video/:key", s.GetVideo)
router.GET("/file/:key", s.GetFile)
router.GET("/voice/:key", s.GetVoice)
router.GET("/data/*path", s.GetMediaData)
// MCP Server
@@ -75,6 +77,8 @@ func (s *Service) GetChatlog(c *gin.Context) {
q := struct {
Time string `form:"time"`
Talker string `form:"talker"`
Sender string `form:"sender"`
Keyword string `form:"keyword"`
Limit int `form:"limit"`
Offset int `form:"offset"`
Format string `form:"format"`
@@ -98,7 +102,7 @@ func (s *Service) GetChatlog(c *gin.Context) {
q.Offset = 0
}
messages, err := s.db.GetMessages(start, end, q.Talker, q.Limit, q.Offset)
messages, err := s.db.GetMessages(start, end, q.Talker, q.Sender, q.Keyword, q.Limit, q.Offset)
if err != nil {
errors.Err(c, err)
return
@@ -117,7 +121,7 @@ func (s *Service) GetChatlog(c *gin.Context) {
c.Writer.Flush()
for _, m := range messages {
c.Writer.WriteString(m.PlainText(len(q.Talker) == 0, c.Request.Host))
c.Writer.WriteString(m.PlainText(strings.Contains(q.Talker, ","), util.PerfectTimeFormat(start, end), c.Request.Host))
c.Writer.WriteString("\n")
c.Writer.Flush()
}
@@ -127,7 +131,7 @@ func (s *Service) GetChatlog(c *gin.Context) {
func (s *Service) GetContacts(c *gin.Context) {
q := struct {
Key string `form:"key"`
Keyword string `form:"keyword"`
Limit int `form:"limit"`
Offset int `form:"offset"`
Format string `form:"format"`
@@ -138,7 +142,7 @@ func (s *Service) GetContacts(c *gin.Context) {
return
}
list, err := s.db.GetContacts(q.Key, q.Limit, q.Offset)
list, err := s.db.GetContacts(q.Keyword, q.Limit, q.Offset)
if err != nil {
errors.Err(c, err)
return
@@ -172,7 +176,7 @@ func (s *Service) GetContacts(c *gin.Context) {
func (s *Service) GetChatRooms(c *gin.Context) {
q := struct {
Key string `form:"key"`
Keyword string `form:"keyword"`
Limit int `form:"limit"`
Offset int `form:"offset"`
Format string `form:"format"`
@@ -183,7 +187,7 @@ func (s *Service) GetChatRooms(c *gin.Context) {
return
}
list, err := s.db.GetChatRooms(q.Key, q.Limit, q.Offset)
list, err := s.db.GetChatRooms(q.Keyword, q.Limit, q.Offset)
if err != nil {
errors.Err(c, err)
return
@@ -216,7 +220,7 @@ func (s *Service) GetChatRooms(c *gin.Context) {
func (s *Service) GetSessions(c *gin.Context) {
q := struct {
Key string `form:"key"`
Keyword string `form:"keyword"`
Limit int `form:"limit"`
Offset int `form:"offset"`
Format string `form:"format"`
@@ -227,7 +231,7 @@ func (s *Service) GetSessions(c *gin.Context) {
return
}
sessions, err := s.db.GetSessions(q.Key, q.Limit, q.Offset)
sessions, err := s.db.GetSessions(q.Keyword, q.Limit, q.Offset)
if err != nil {
errors.Err(c, err)
return
@@ -272,6 +276,9 @@ func (s *Service) GetVideo(c *gin.Context) {
func (s *Service) GetFile(c *gin.Context) {
s.GetMedia(c, "file")
}
func (s *Service) GetVoice(c *gin.Context) {
s.GetMedia(c, "voice")
}
func (s *Service) GetMedia(c *gin.Context, _type string) {
key := c.Param("key")
@@ -291,7 +298,13 @@ func (s *Service) GetMedia(c *gin.Context, _type string) {
return
}
switch media.Type {
case "voice":
s.HandleVoice(c, media.Data)
default:
c.Redirect(http.StatusFound, "/data/"+media.Path)
}
}
func (s *Service) GetMediaData(c *gin.Context) {
@@ -343,3 +356,12 @@ func (s *Service) HandleDatFile(c *gin.Context, path string) {
c.File(path)
}
}
func (s *Service) HandleVoice(c *gin.Context, data []byte) {
out, err := silk.Silk2MP3(data)
if err != nil {
c.Data(http.StatusOK, "audio/silk", data)
return
}
c.Data(http.StatusOK, "audio/mp3", out)
}

View File

@@ -77,6 +77,21 @@ func (s *Service) Start() error {
return nil
}
func (s *Service) ListenAndServe() error {
if s.ctx.HTTPAddr == "" {
s.ctx.HTTPAddr = DefalutHTTPAddr
}
s.server = &http.Server{
Addr: s.ctx.HTTPAddr,
Handler: s.router,
}
log.Info().Msg("Starting HTTP server on " + s.ctx.HTTPAddr)
return s.server.ListenAndServe()
}
func (s *Service) Stop() error {
if s.server == nil {
@@ -84,11 +99,12 @@ func (s *Service) Stop() error {
}
// 使用超时上下文优雅关闭
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()
if err := s.server.Shutdown(ctx); err != nil {
return errors.HTTPShutDown(err)
log.Debug().Err(err).Msg("Failed to shutdown HTTP server")
return nil
}
log.Info().Msg("HTTP server stopped")

View File

@@ -1,156 +1,757 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Chatlog</title>
<style>
.random-paragraph {
:root {
--primary-color: #3498db;
--primary-dark: #2980b9;
--success-color: #2ecc71;
--success-dark: #27ae60;
--error-color: #e74c3c;
--bg-light: #f5f5f5;
--bg-white: #ffffff;
--text-color: #333333;
--border-color: #dddddd;
}
body {
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto,
Oxygen, Ubuntu, Cantarell, "Open Sans", "Helvetica Neue", sans-serif;
line-height: 1.6;
color: var(--text-color);
max-width: 1200px;
margin: 0 auto;
padding: 20px;
background-color: #fafafa;
}
.container {
display: flex;
flex-direction: column;
align-items: center;
width: 100%;
}
.welcome-text {
text-align: center;
margin-bottom: 30px;
}
.api-section {
background-color: var(--bg-light);
border-radius: 10px;
padding: 25px;
width: 100%;
max-width: 850px;
margin-bottom: 30px;
box-shadow: 0 2px 10px rgba(0, 0, 0, 0.05);
}
h1 {
color: #2c3e50;
margin-bottom: 15px;
}
h2 {
color: var(--primary-color);
margin-top: 20px;
border-bottom: 2px solid var(--primary-color);
padding-bottom: 8px;
display: inline-block;
}
h3 {
margin-top: 20px;
color: #34495e;
}
.docs-link {
color: var(--primary-color);
text-decoration: none;
font-weight: bold;
transition: all 0.2s ease;
}
.docs-link:hover {
text-decoration: underline;
color: var(--primary-dark);
}
.api-tester {
background-color: var(--bg-white);
border: 1px solid var(--border-color);
border-radius: 10px;
padding: 25px;
margin-top: 20px;
box-shadow: 0 2px 8px rgba(0, 0, 0, 0.03);
}
.form-group {
margin-bottom: 18px;
}
label {
display: block;
margin-bottom: 6px;
font-weight: 600;
color: #34495e;
}
input,
select,
textarea {
width: 100%;
padding: 10px 12px;
border: 1px solid #ddd;
border-radius: 6px;
box-sizing: border-box;
font-size: 14px;
transition: all 0.3s;
}
input:focus,
select:focus,
textarea:focus {
outline: none;
border-color: var(--primary-color);
box-shadow: 0 0 0 2px rgba(52, 152, 219, 0.2);
}
input::placeholder,
textarea::placeholder {
color: #aaa;
}
button {
background-color: var(--primary-color);
color: white;
border: none;
padding: 12px 18px;
border-radius: 6px;
cursor: pointer;
font-size: 16px;
font-weight: 500;
transition: all 0.3s;
display: inline-flex;
align-items: center;
justify-content: center;
}
button:hover {
background-color: var(--primary-dark);
transform: translateY(-1px);
box-shadow: 0 2px 5px rgba(0, 0, 0, 0.1);
}
button:active {
transform: translateY(0);
}
.result-container {
margin-top: 20px;
border: 1px solid var(--border-color);
border-radius: 6px;
padding: 15px;
background-color: #f9f9f9;
max-height: 400px;
overflow-y: auto;
white-space: pre-wrap;
font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo,
monospace;
font-size: 14px;
line-height: 1.5;
position: relative;
}
.request-url {
background-color: #f0f0f0;
padding: 10px;
border-radius: 6px;
margin-bottom: 10px;
font-family: "SFMono-Regular", Consolas, "Liberation Mono", Menlo,
monospace;
font-size: 14px;
word-break: break-all;
border: 1px dashed #ccc;
display: flex;
justify-content: space-between;
align-items: center;
}
.url-text {
flex-grow: 1;
margin-right: 10px;
}
.copy-url-button {
background-color: #9b59b6;
padding: 6px 12px;
font-size: 12px;
white-space: nowrap;
}
.loading {
text-align: center;
padding: 20px;
color: #666;
}
.loading::after {
content: "...";
animation: dots 1.5s steps(5, end) infinite;
}
@keyframes dots {
0%,
20% {
content: ".";
}
40% {
content: "..";
}
60% {
content: "...";
}
80%,
100% {
content: "";
}
}
.tab-container {
display: flex;
margin-bottom: 20px;
border-bottom: 1px solid #e0e0e0;
}
.tab {
padding: 12px 20px;
cursor: pointer;
margin-right: 5px;
border-radius: 6px 6px 0 0;
font-weight: 500;
transition: all 0.2s;
border: 1px solid transparent;
border-bottom: none;
position: relative;
bottom: -1px;
}
.tab:hover {
background-color: #f0f8ff;
}
.tab.active {
background-color: var(--bg-white);
border-color: #e0e0e0;
color: var(--primary-color);
border-bottom: 1px solid white;
}
.tab-content {
display: none;
padding: 20px 0;
}
.tab-content.active {
display: block;
animation: fadeIn 0.3s;
}
@keyframes fadeIn {
from {
opacity: 0;
}
to {
opacity: 1;
}
}
.button-group {
display: flex;
justify-content: flex-end;
margin-top: 20px;
}
.copy-button {
background-color: var(--success-color);
padding: 8px 15px;
font-size: 14px;
margin-left: 10px;
}
.copy-button:hover {
background-color: var(--success-dark);
}
.error-message {
color: var(--error-color);
font-weight: bold;
margin-top: 10px;
padding: 10px;
border-radius: 4px;
background-color: rgba(231, 76, 60, 0.1);
border-left: 4px solid var(--error-color);
display: none;
}
</style>
</head>
<body>
<div id="paragraphContainer">
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
('-. .-. ('-. .-') _
( OO ) / ( OO ).-. ( OO) )
.-----. ,--. ,--. / . --. / / '._ ,--. .-'),-----. ,----.
' .--./ | | | | | \-. \ |'--...__) | |.-') ( OO' .-. ' ' .-./-')
| |('-. | .| | .-'-' | | '--. .--' | | OO ) / | | | | | |_( O- )
/_) |OO ) | | \| |_.' | | | | |`-' | \_) | |\| | | | .--, \
|| |`-'| | .-. | | .-. | | | (| '---.' \ | | | |(| | '. (_/
(_' '--'\ | | | | | | | | | | | | `' '-' ' | '--' |
`-----' `--' `--' `--' `--' `--' `------' `-----' `------'
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
_____ _ _ _
/ __ \| | | | | |
| / \/| |__ __ _ | |_ | | ___ __ _
| | | '_ \ / _` || __|| | / _ \ / _` |
| \__/\| | | || (_| || |_ | || (_) || (_| |
\____/|_| |_| \__,_| \__||_| \___/ \__, |
__/ |
|___/
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
,-----. ,--. ,--. ,--.
' .--./ | ,---. ,--,--. ,-' '-. | | ,---. ,---.
| | | .-. | ' ,-. | '-. .-' | | | .-. | | .-. |
' '--'\ | | | | \ '-' | | | | | ' '-' ' ' '-' '
`-----' `--' `--' `--`--' `--' `--' `---' .`- /
`---'
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
____ _ _ _
/ ___| | |__ __ _ | |_ | | ___ __ _
| | | '_ \ / _` | | __| | | / _ \ / _` |
| |___ | | | | | (_| | | |_ | | | (_) | | (_| |
\____| |_| |_| \__,_| \__| |_| \___/ \__, |
|___/
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
_____ _____ _____ _____ _____ _______ _____
/\ \ /\ \ /\ \ /\ \ /\ \ /::\ \ /\ \
/::\ \ /::\____\ /::\ \ /::\ \ /::\____\ /::::\ \ /::\ \
/::::\ \ /:::/ / /::::\ \ \:::\ \ /:::/ / /::::::\ \ /::::\ \
/::::::\ \ /:::/ / /::::::\ \ \:::\ \ /:::/ / /::::::::\ \ /::::::\ \
/:::/\:::\ \ /:::/ / /:::/\:::\ \ \:::\ \ /:::/ / /:::/~~\:::\ \ /:::/\:::\ \
/:::/ \:::\ \ /:::/____/ /:::/__\:::\ \ \:::\ \ /:::/ / /:::/ \:::\ \ /:::/ \:::\ \
/:::/ \:::\ \ /::::\ \ /::::\ \:::\ \ /::::\ \ /:::/ / /:::/ / \:::\ \ /:::/ \:::\ \
/:::/ / \:::\ \ /::::::\ \ _____ /::::::\ \:::\ \ /::::::\ \ /:::/ / /:::/____/ \:::\____\ /:::/ / \:::\ \
/:::/ / \:::\ \ /:::/\:::\ \ /\ \ /:::/\:::\ \:::\ \ /:::/\:::\ \ /:::/ / |:::| | |:::| | /:::/ / \:::\ ___\
/:::/____/ \:::\____\/:::/ \:::\ /::\____\/:::/ \:::\ \:::\____\ /:::/ \:::\____\/:::/____/ |:::|____| |:::| |/:::/____/ ___\:::| |
\:::\ \ \::/ /\::/ \:::\ /:::/ /\::/ \:::\ /:::/ / /:::/ \::/ /\:::\ \ \:::\ \ /:::/ / \:::\ \ /\ /:::|____|
\:::\ \ \/____/ \/____/ \:::\/:::/ / \/____/ \:::\/:::/ / /:::/ / \/____/ \:::\ \ \:::\ \ /:::/ / \:::\ /::\ \::/ /
\:::\ \ \::::::/ / \::::::/ / /:::/ / \:::\ \ \:::\ /:::/ / \:::\ \:::\ \/____/
\:::\ \ \::::/ / \::::/ / /:::/ / \:::\ \ \:::\__/:::/ / \:::\ \:::\____\
\:::\ \ /:::/ / /:::/ / \::/ / \:::\ \ \::::::::/ / \:::\ /:::/ /
\:::\ \ /:::/ / /:::/ / \/____/ \:::\ \ \::::::/ / \:::\/:::/ /
\:::\ \ /:::/ / /:::/ / \:::\ \ \::::/ / \::::::/ /
\:::\____\ /:::/ / /:::/ / \:::\____\ \::/____/ \::::/ /
\::/ / \::/ / \::/ / \::/ / ~~ \::/____/
\/____/ \/____/ \/____/ \/____/
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
___ _ _ __ ____ __ _____ ___
/ __)( )_( ) /__\ (_ _)( ) ( _ ) / __)
( (__ ) _ ( /(__)\ )( )(__ )(_)( ( (_-.
\___)(_) (_)(__)(__) (__) (____)(_____) \___/
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
________ ___ ___ ________ _________ ___ ________ ________
|\ ____\ |\ \|\ \ |\ __ \ |\___ ___\ |\ \ |\ __ \ |\ ____\
\ \ \___| \ \ \\\ \ \ \ \|\ \ \|___ \ \_| \ \ \ \ \ \|\ \ \ \ \___|
\ \ \ \ \ __ \ \ \ __ \ \ \ \ \ \ \ \ \ \\\ \ \ \ \ ___
\ \ \____ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \____ \ \ \\\ \ \ \ \|\ \
\ \_______\ \ \__\ \__\ \ \__\ \__\ \ \__\ \ \_______\ \ \_______\ \ \_______\
\|_______| \|__|\|__| \|__|\|__| \|__| \|_______| \|_______| \|_______|
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
╔═╗┬ ┬┌─┐┌┬┐┬ ┌─┐┌─┐
║ ├─┤├─┤ │ │ │ ││ ┬
╚═╝┴ ┴┴ ┴ ┴ ┴─┘└─┘└─┘
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
▄▄· ▄ .▄ ▄▄▄· ▄▄▄▄▄▄▄▌ ▄▄ •
▐█ ▌▪██▪▐█▐█ ▀█ •██ ██• ▪ ▐█ ▀ ▪
██ ▄▄██▀▐█▄█▀▀█ ▐█.▪██▪ ▄█▀▄ ▄█ ▀█▄
▐███▌██▌▐▀▐█ ▪▐▌ ▐█▌·▐█▌▐▌▐█▌.▐▌▐█▄▪▐█
·▀▀▀ ▀▀▀ · ▀ ▀ ▀▀▀ .▀▀▀ ▀█▄▀▪·▀▀▀▀
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
,. - ., .·¨'`; ,.·´¨;\ ,., ' , . ., ° ,. ' , ·. ,.-·~·., ,.-·^*ª'` ·,
,·'´ ,. - , ';\ '; ;'\ '; ;::\ ;´ '· ., ;'´ , ., _';\' / ';\ / ·'´,.-·-., `,' .·´ ,·'´:¯'`·, '\
,·´ .'´\:::::;' ;:'\ ' ; ;::'\ ,' ;::'; .´ .-, ';\ \:´¨¯:;' `;::'\:'\ ,' ,'::'\ / .'´\:::::::'\ '\ ° ,´ ,'\:::::::::\,.·\'
/ ,'´::::'\;:-/ ,' ::; ' ; ;::_';,. ,.' ;:::';° / /:\:'; ;:'\' \::::; ,'::_'\;' ,' ;:::';' ,·' ,'::::\:;:-·-:'; ';\ / /:::\;·'´¯'`·;\:::\°
,' ;':::::;'´ '; /\::;' ' .' ,. -·~-·, ;:::'; ' ,' ,'::::'\'; ;::'; ,' ,'::;' '; ,':::;' ;. ';:::;´ ,' ,':'\ ; ;:::;' '\;:·´
; ;:::::; '\*'´\::\' ° '; ;'\::::::::; '/::::; ,.-·' '·~^*'´¨, ';::; ; ;:::; ° ; ,':::;' ' '; ;::; ,'´ .'´\::'; '; ;::/ ,·´¯'; °
'; ';::::'; '\::'\/.' ; ';:;\;::-··; ;::::; ':, ,·:²*´¨¯'`; ;::'; ; ;::;' ,' ,'::;' '; ':;: ,.·´,.·´::::\;'° '; '·;' ,.·´, ;'\
\ '·:;:'_ ,. -·'´.·´\ ':,.·´\;' ;' ,' :::/ ' ,' / \::::::::'; ;::'; ; ;::;' ; ';_:,.-·´';\ \·, `*´,.·'´::::::;·´ \'·. `'´,.·:´'; ;::\'
'\:` · .,. -·:´::::::\' \:::::\ \·.'::::; ,' ,'::::\·²*'´¨¯':,'\:; ',.'\::;' ', _,.-·'´:\:\ \\:¯::\:::::::;:·´ '\::\¯::::::::'; ;::';
\:::::::\:::::::;:·'´' \;:·´ \:\::'; \`¨\:::/ \::\' \::\:;' \¨:::::::::::\'; `\:::::\;::·'´ ° `·:\:::;:·´';.·´\::;'
`· :;::\;::-·´ `·\;' '\::\;' '\;' ' \;:' '\;::_;:-·'´ ¯ ¯ \::::\;'
' `¨' ° '¨ '\:·´'
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
▄████▄ ██░ ██ ▄▄▄ ▄▄▄█████▓ ██▓ ▒█████ ▄████
▒██▀ ▀█ ▓██░ ██▒▒████▄ ▓ ██▒ ▓▒▓██▒ ▒██▒ ██▒ ██▒ ▀█▒
▒▓█ ▄ ▒██▀▀██░▒██ ▀█▄ ▒ ▓██░ ▒░▒██░ ▒██░ ██▒▒██░▄▄▄░
▒▓▓▄ ▄██▒░▓█ ░██ ░██▄▄▄▄██ ░ ▓██▓ ░ ▒██░ ▒██ ██░░▓█ ██▓
▒ ▓███▀ ░░▓█▒░██▓ ▓█ ▓██▒ ▒██▒ ░ ░██████▒░ ████▓▒░░▒▓███▀▒
░ ░▒ ▒ ░ ▒ ░░▒░▒ ▒▒ ▓▒█░ ▒ ░░ ░ ▒░▓ ░░ ▒░▒░▒░ ░▒ ▒
░ ▒ ▒ ░▒░ ░ ▒ ▒▒ ░ ░ ░ ░ ▒ ░ ░ ▒ ▒░ ░ ░
░ ░ ░░ ░ ░ ▒ ░ ░ ░ ░ ░ ░ ▒ ░ ░ ░
░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░ ░
</pre>
<pre style="float:left;white-space:pre-wrap;" class="random-paragraph">
▄█▄ ▄ █ ██ ▄▄▄▄▀ █ ████▄ ▄▀
█▀ ▀▄ █ █ █ █ ▀▀▀ █ █ █ █ ▄▀
█ ▀ ██▀▀█ █▄▄█ █ █ █ █ █ ▀▄
█▄ ▄▀ █ █ █ █ █ ███▄ ▀████ █ █
▀███▀ █ █ ▀ ▀ ███
▀ █
</pre>
</div>
<script>
window.onload = function() {
showRandomParagraph();
};
function showRandomParagraph() {
const paragraphs = document.getElementsByClassName("random-paragraph");
for (let i = 0; i < paragraphs.length; i++) {
paragraphs[i].style.display = "none";
.api-description {
margin-bottom: 15px;
color: #555;
}
const randomIndex = Math.floor(Math.random() * paragraphs.length);
paragraphs[randomIndex].style.display = "block";
.badge {
display: inline-block;
padding: 3px 8px;
border-radius: 12px;
font-size: 12px;
font-weight: 600;
margin-left: 8px;
background-color: rgba(52, 152, 219, 0.1);
color: var(--primary-color);
}
.optional-param {
font-size: 12px;
color: #777;
margin-left: 5px;
font-style: italic;
}
.required-field {
color: var(--error-color);
font-weight: bold;
}
</style>
</head>
<body>
<div class="container">
<div class="welcome-text">
<h1>🎉 恭喜Chatlog 服务已成功启动</h1>
<p>
Chatlog 是一个帮助你轻松使用自己聊天数据的工具,现在你可以通过 HTTP
API 访问你的聊天记录、联系人和群聊信息。
</p>
</div>
<div class="api-section">
<h2>🔍 API 接口与调试</h2>
<div class="api-tester">
<div class="tab-container">
<div class="tab active" data-tab="session">最近会话</div>
<div class="tab" data-tab="chatroom">群聊</div>
<div class="tab" data-tab="contact">联系人</div>
<div class="tab" data-tab="chatlog">聊天记录</div>
</div>
<!-- 会话查询表单 -->
<div class="tab-content active" id="session-tab">
<div class="api-description">
<p>
查询最近会话列表。<span class="badge">GET /api/v1/session</span>
</p>
</div>
<div class="form-group">
<label for="session-format"
>输出格式:<span class="optional-param">可选</span></label
>
<select id="session-format">
<option value="">默认</option>
<option value="json">JSON</option>
<option value="text">纯文本</option>
</select>
</div>
</div>
<!-- 群聊查询表单 -->
<div class="tab-content" id="chatroom-tab">
<div class="api-description">
<p>
查询群聊列表,可选择性地按关键词搜索。<span class="badge"
>GET /api/v1/chatroom</span
>
</p>
</div>
<div class="form-group">
<label for="chatroom-keyword"
>搜索群聊:<span class="optional-param">可选</span></label
>
<input
type="text"
id="chatroom-keyword"
placeholder="输入关键词搜索群聊"
/>
</div>
<div class="form-group">
<label for="chatroom-format"
>输出格式:<span class="optional-param">可选</span></label
>
<select id="chatroom-format">
<option value="">默认</option>
<option value="json">JSON</option>
<option value="text">纯文本</option>
</select>
</div>
</div>
<!-- 联系人查询表单 -->
<div class="tab-content" id="contact-tab">
<div class="api-description">
<p>
查询联系人列表,可选择性地按关键词搜索。<span class="badge"
>GET /api/v1/contact</span
>
</p>
</div>
<div class="form-group">
<label for="contact-keyword"
>搜索联系人:<span class="optional-param">可选</span></label
>
<input
type="text"
id="contact-keyword"
placeholder="输入关键词搜索联系人"
/>
</div>
<div class="form-group">
<label for="contact-format"
>输出格式:<span class="optional-param">可选</span></label
>
<select id="contact-format">
<option value="">默认</option>
<option value="json">JSON</option>
<option value="text">纯文本</option>
</select>
</div>
</div>
<!-- 聊天记录查询表单 -->
<div class="tab-content" id="chatlog-tab">
<div class="api-description">
<p>
查询指定时间范围内与特定联系人或群聊的聊天记录。<span
class="badge"
>GET /api/v1/chatlog</span
>
</p>
</div>
<div class="form-group">
<label for="time"
>时间范围:<span class="required-field">*</span></label
>
<input
type="text"
id="time"
placeholder="例如2023-01-01 或 2023-01-01~2023-01-31"
/>
</div>
<div class="form-group">
<label for="talker"
>聊天对象:<span class="required-field">*</span></label
>
<input
type="text"
id="talker"
placeholder="wxid、群ID、备注名或昵称"
/>
</div>
<div class="form-group">
<label for="sender"
>发送者:<span class="optional-param">可选</span></label
>
<input
type="text"
id="sender"
placeholder="指定消息发送者"
/>
</div>
<div class="form-group">
<label for="keyword"
>关键词:<span class="optional-param">可选</span></label
>
<input
type="text"
id="keyword"
placeholder="搜索消息内容中的关键词"
/>
</div>
<div class="form-group">
<label for="limit"
>返回数量:<span class="optional-param">可选</span></label
>
<input type="number" id="limit" placeholder="默认不做限制" />
</div>
<div class="form-group">
<label for="offset"
>偏移量:<span class="optional-param">可选</span></label
>
<input type="number" id="offset" placeholder="默认 0" />
</div>
<div class="form-group">
<label for="format"
>输出格式:<span class="optional-param">可选</span></label
>
<select id="format">
<option value="">默认</option>
<option value="text">纯文本</option>
<option value="json">JSON</option>
<option value="csv">CSV</option>
</select>
</div>
</div>
<button id="test-api">执行查询</button>
<div id="result-wrapper" style="display: none; margin-top: 20px">
<div class="request-url" id="request-url-container">
<span class="url-text" id="request-url"></span>
<button class="copy-button copy-url-button" id="copy-url">
复制请求URL
</button>
</div>
<div class="result-container" id="api-result">
<p>查询结果将显示在这里...</p>
</div>
<div class="button-group">
<button class="copy-button" id="copy-result">复制结果</button>
</div>
</div>
<div class="error-message" id="error-message"></div>
</div>
</div>
<div class="api-section">
<h2>🤖 MCP 集成</h2>
<p>
Chatlog 支持 MCP (Model Context Protocol) SSE 协议,可与支持 MCP 的 AI
助手无缝集成。
</p>
<p>SSE 端点:<strong>/sse</strong></p>
<p>
详细集成指南请参考
<a
href="https://github.com/sjzar/chatlog/blob/main/docs/mcp.md"
class="docs-link"
target="_blank"
>MCP 集成指南</a
>
</p>
</div>
<div class="api-section">
<h2>📚 更多资源</h2>
<p>
查看
<a
href="https://github.com/sjzar/chatlog"
class="docs-link"
target="_blank"
>GitHub 项目</a
>
获取完整文档和使用指南。
</p>
<p>
如果你有任何问题或建议,欢迎通过
<a
href="https://github.com/sjzar/chatlog/discussions"
class="docs-link"
target="_blank"
>Discussions</a
>
进行交流。
</p>
</div>
</div>
<script>
// 标签切换功能
document.querySelectorAll(".tab").forEach((tab) => {
tab.addEventListener("click", function () {
// 移除所有标签的活动状态
document
.querySelectorAll(".tab")
.forEach((t) => t.classList.remove("active"));
// 设置当前标签为活动状态
this.classList.add("active");
// 隐藏所有内容区域
document.querySelectorAll(".tab-content").forEach((content) => {
content.classList.remove("active");
});
// 显示当前标签对应的内容
const tabId = this.getAttribute("data-tab") + "-tab";
document.getElementById(tabId).classList.add("active");
// 清空结果区域
document.getElementById("result-wrapper").style.display = "none";
document.getElementById("api-result").innerHTML =
"<p>查询结果将显示在这里...</p>";
document.getElementById("request-url").textContent = "";
document.getElementById("error-message").style.display = "none";
document.getElementById("error-message").textContent = "";
});
});
// API 测试功能
document
.getElementById("test-api")
.addEventListener("click", async function () {
const resultContainer = document.getElementById("api-result");
const requestUrlContainer = document.getElementById("request-url");
const errorMessage = document.getElementById("error-message");
const resultWrapper = document.getElementById("result-wrapper");
errorMessage.style.display = "none";
errorMessage.textContent = "";
try {
// 获取当前活动的标签
const activeTab = document
.querySelector(".tab.active")
.getAttribute("data-tab");
let url = "/api/v1/";
let params = new URLSearchParams();
// 根据不同的标签构建不同的请求
switch (activeTab) {
case "chatlog":
url += "chatlog";
const time = document.getElementById("time").value;
const talker = document.getElementById("talker").value;
const sender = document.getElementById("sender").value;
const keyword = document.getElementById("keyword").value;
const limit = document.getElementById("limit").value;
const offset = document.getElementById("offset").value;
const format = document.getElementById("format").value;
// 验证必填项
if (!time || !talker) {
errorMessage.textContent =
"错误: 时间范围和聊天对象为必填项!";
errorMessage.style.display = "block";
return;
}
if (time) params.append("time", time);
if (talker) params.append("talker", talker);
if (sender) params.append("sender", sender);
if (keyword) params.append("keyword", keyword);
if (limit) params.append("limit", limit);
if (offset) params.append("offset", offset);
if (format) params.append("format", format);
break;
case "contact":
url += "contact";
const contactKeyword =
document.getElementById("contact-keyword").value;
const contactFormat =
document.getElementById("contact-format").value;
if (contactKeyword) params.append("keyword", contactKeyword);
if (contactFormat) params.append("format", contactFormat);
break;
case "chatroom":
url += "chatroom";
const chatroomKeyword =
document.getElementById("chatroom-keyword").value;
const chatroomFormat =
document.getElementById("chatroom-format").value;
if (chatroomKeyword) params.append("keyword", chatroomKeyword);
if (chatroomFormat) params.append("format", chatroomFormat);
break;
case "session":
url += "session";
const sessionFormat =
document.getElementById("session-format").value;
if (sessionFormat) params.append("format", sessionFormat);
break;
}
// 添加参数到URL
const apiUrl = params.toString()
? `${url}?${params.toString()}`
: url;
// 获取完整URL包含域名部分
const fullUrl = window.location.origin + apiUrl;
// 显示完整请求URL
requestUrlContainer.textContent = fullUrl;
resultWrapper.style.display = "block";
// 显示加载中
resultContainer.innerHTML = '<div class="loading">加载中</div>';
// 发送请求
const response = await fetch(apiUrl);
if (!response.ok) {
throw new Error(`HTTP error! Status: ${response.status}`);
}
// 获取响应内容类型
const contentType = response.headers.get("content-type");
let result;
if (contentType && contentType.includes("application/json")) {
// 如果是JSON格式化显示
result = await response.json();
resultContainer.innerHTML = JSON.stringify(result, null, 2);
} else {
// 其他格式直接显示文本
result = await response.text();
resultContainer.innerHTML = result;
}
} catch (error) {
resultContainer.innerHTML = "";
errorMessage.textContent = `查询出错: ${error.message}`;
errorMessage.style.display = "block";
console.error("API查询出错:", error);
}
});
// 复制结果功能
document
.getElementById("copy-result")
.addEventListener("click", function () {
const resultText = document.getElementById("api-result").innerText;
copyToClipboard(resultText, this, "已复制结果!");
});
// 复制URL功能
document
.getElementById("copy-url")
.addEventListener("click", function () {
// 获取完整URL包含域名部分
const urlText = document.getElementById("request-url").innerText;
copyToClipboard(urlText, this, "已复制URL!");
});
// 通用复制功能
function copyToClipboard(text, button, successMessage) {
navigator.clipboard
.writeText(text)
.then(() => {
const originalText = button.textContent;
button.textContent = successMessage;
setTimeout(() => {
button.textContent = originalText;
}, 2000);
})
.catch((err) => {
console.error("复制失败:", err);
});
}
</script>
</body>
</body>
</html>

View File

@@ -6,12 +6,14 @@ import (
"path/filepath"
"strings"
"github.com/rs/zerolog/log"
"github.com/sjzar/chatlog/internal/chatlog/conf"
"github.com/sjzar/chatlog/internal/chatlog/ctx"
"github.com/sjzar/chatlog/internal/chatlog/database"
"github.com/sjzar/chatlog/internal/chatlog/http"
"github.com/sjzar/chatlog/internal/chatlog/mcp"
"github.com/sjzar/chatlog/internal/chatlog/wechat"
iwechat "github.com/sjzar/chatlog/internal/wechat"
"github.com/sjzar/chatlog/pkg/util"
"github.com/sjzar/chatlog/pkg/util/dat2img"
)
@@ -79,6 +81,33 @@ func (m *Manager) Run() error {
return nil
}
func (m *Manager) Switch(info *iwechat.Account, history string) error {
if m.ctx.AutoDecrypt {
if err := m.StopAutoDecrypt(); err != nil {
return err
}
}
if m.ctx.HTTPEnabled {
if err := m.stopService(); err != nil {
return err
}
}
if info != nil {
m.ctx.SwitchCurrent(info)
} else {
m.ctx.SwitchHistory(history)
}
if m.ctx.HTTPEnabled {
// 启动HTTP服务
if err := m.StartService(); err != nil {
log.Info().Err(err).Msg("启动服务失败")
m.StopService()
}
}
return nil
}
func (m *Manager) StartService() error {
// 按依赖顺序启动服务
@@ -109,6 +138,17 @@ func (m *Manager) StartService() error {
}
func (m *Manager) StopService() error {
if err := m.stopService(); err != nil {
return err
}
// 更新状态
m.ctx.SetHTTPEnabled(false)
return nil
}
func (m *Manager) stopService() error {
// 按依赖的反序停止服务
var errs []error
@@ -124,9 +164,6 @@ func (m *Manager) StopService() error {
errs = append(errs, err)
}
// 更新状态
m.ctx.SetHTTPEnabled(false)
// 如果有错误,返回第一个错误
if len(errs) > 0 {
return errs[0]
@@ -138,7 +175,7 @@ func (m *Manager) StopService() error {
func (m *Manager) SetHTTPAddr(text string) error {
var addr string
if util.IsNumeric(text) {
addr = fmt.Sprintf("0.0.0.0:%s", text)
addr = fmt.Sprintf("127.0.0.1:%s", text)
} else if strings.HasPrefix(text, "http://") {
addr = strings.TrimPrefix(text, "http://")
} else if strings.HasPrefix(text, "https://") {
@@ -175,7 +212,7 @@ func (m *Manager) DecryptDBFiles() error {
m.ctx.WorkDir = util.DefaultWorkDir(m.ctx.Account)
}
if err := m.wechat.DecryptDBFiles(m.ctx.DataDir, m.ctx.WorkDir, m.ctx.DataKey, m.ctx.Platform, m.ctx.Version); err != nil {
if err := m.wechat.DecryptDBFiles(); err != nil {
return err
}
m.ctx.Refresh()
@@ -183,6 +220,48 @@ func (m *Manager) DecryptDBFiles() error {
return nil
}
func (m *Manager) StartAutoDecrypt() error {
if m.ctx.DataKey == "" || m.ctx.DataDir == "" {
return fmt.Errorf("请先获取密钥")
}
if m.ctx.WorkDir == "" {
return fmt.Errorf("请先执行解密数据")
}
if err := m.wechat.StartAutoDecrypt(); err != nil {
return err
}
m.ctx.SetAutoDecrypt(true)
return nil
}
func (m *Manager) StopAutoDecrypt() error {
if err := m.wechat.StopAutoDecrypt(); err != nil {
return err
}
m.ctx.SetAutoDecrypt(false)
return nil
}
func (m *Manager) RefreshSession() error {
if m.db.GetDB() == nil {
if err := m.db.Start(); err != nil {
return err
}
}
resp, err := m.db.GetSessions("", 1, 0)
if err != nil {
return err
}
if len(resp.Items) == 0 {
return nil
}
m.ctx.LastSession = resp.Items[0].NTime
return nil
}
func (m *Manager) CommandKey(pid int) (string, error) {
instances := m.wechat.GetWeChatInstances()
if len(instances) == 0 {
@@ -216,10 +295,55 @@ func (m *Manager) CommandDecrypt(dataDir string, workDir string, key string, pla
if workDir == "" {
workDir = util.DefaultWorkDir(filepath.Base(filepath.Dir(dataDir)))
}
if err := m.wechat.DecryptDBFiles(dataDir, workDir, key, platform, version); err != nil {
m.ctx.DataDir = dataDir
m.ctx.WorkDir = workDir
m.ctx.DataKey = key
m.ctx.Platform = platform
m.ctx.Version = version
if err := m.wechat.DecryptDBFiles(); err != nil {
return err
}
return nil
}
func (m *Manager) CommandHTTPServer(addr string, dataDir string, workDir string, platform string, version int) error {
if addr == "" {
addr = "127.0.0.1:5030"
}
if workDir == "" {
return fmt.Errorf("workDir is required")
}
if platform == "" {
return fmt.Errorf("platform is required")
}
if version == 0 {
return fmt.Errorf("version is required")
}
m.ctx.HTTPAddr = addr
m.ctx.DataDir = dataDir
m.ctx.WorkDir = workDir
m.ctx.Platform = platform
m.ctx.Version = version
// 如果是 4.0 版本,更新下 xorkey
if m.ctx.Version == 4 && m.ctx.DataDir != "" {
go dat2img.ScanAndSetXorKey(m.ctx.DataDir)
}
// 按依赖顺序启动服务
if err := m.db.Start(); err != nil {
return err
}
if err := m.mcp.Start(); err != nil {
return err
}
return m.http.ListenAndServe()
}

View File

@@ -21,12 +21,12 @@ var (
InputSchema: mcp.ToolSchema{
Type: "object",
Properties: mcp.M{
"query": mcp.M{
"keyword": mcp.M{
"type": "string",
"description": "联系人的搜索关键词可以是姓名、备注名或ID。",
},
},
Required: []string{"query"},
Required: []string{"keyword"},
},
}
@@ -36,12 +36,12 @@ var (
InputSchema: mcp.ToolSchema{
Type: "object",
Properties: mcp.M{
"query": mcp.M{
"keyword": mcp.M{
"type": "string",
"description": "群聊的搜索关键词可以是群名称、群ID或相关描述",
},
},
Required: []string{"query"},
Required: []string{"keyword"},
},
}
@@ -56,23 +56,135 @@ var (
ToolChatLog = mcp.Tool{
Name: "chatlog",
Description: "查询特定时间或时间段内与特定联系人或群组的聊天记录。当用户需要回顾过去的对话内容、查找特定信息或想了解与某人/某群的历史交流时使用此工具。",
Description: `检索历史聊天记录,可根据时间、对话方、发送者和关键词等条件进行精确查询。当用户需要查找特定信息或想了解与某人/某群的历史交流时使用此工具。
【强制多步查询流程!】
当查询特定话题或特定发送者发言时,必须严格按照以下流程使用,任何偏离都会导致错误的结果:
步骤1: 初步定位相关消息
- 使用keyword参数查找特定话题
- 使用sender参数查找特定发送者的消息
- 使用较宽时间范围初步查询
步骤2: 【必须执行】针对每个关键结果点分别获取上下文
- 必须对步骤1返回的每个时间点T1, T2, T3...分别执行独立查询(时间范围接近的消息可以合并为一个查询)
- 每次独立查询必须移除keyword参数
- 每次独立查询必须移除sender参数
- 每次独立查询使用"Tn前后15-30分钟"的窄范围
- 每次独立查询仅保留talker参数
步骤3: 【必须执行】综合分析所有上下文
- 必须等待所有步骤2的查询结果返回后再进行分析
- 必须综合考虑所有上下文信息后再回答用户
【严格执行规则!】
- 禁止仅凭步骤1的结果直接回答用户
- 禁止在步骤2使用过大的时间范围一次性查询所有上下文
- 禁止跳过步骤2或步骤3
- 必须对每个关键结果点分别执行独立的上下文查询
【执行示例】
正确流程示例:
1. 步骤1: chatlog(time="2023-04-01~2023-04-30", talker="工作群", keyword="项目进度")
返回结果: 4月5日、4月12日、4月20日有相关消息
2. 步骤2:
- 查询1: chatlog(time="2023-04-05/09:30~2023-04-05/10:30", talker="工作群") // 注意没有keyword
- 查询2: chatlog(time="2023-04-12/14:00~2023-04-12/15:00", talker="工作群") // 注意没有keyword
- 查询3: chatlog(time="2023-04-20/16:00~2023-04-20/17:00", talker="工作群") // 注意没有keyword
3. 步骤3: 综合分析所有上下文后回答用户
错误流程示例:
- 仅执行步骤1后直接回答
- 步骤2使用time="2023-04-01~2023-04-30"一次性查询
- 步骤2仍然保留keyword或sender参数
【自我检查】回答用户前必须自问:
- 我是否对每个关键时间点都执行了独立的上下文查询?
- 我是否在上下文查询中移除了keyword和sender参数?
- 我是否分析了所有上下文后再回答?
- 如果上述任一问题答案为"否",则必须纠正流程
返回格式:"昵称(ID) 时间\n消息内容\n昵称(ID) 时间\n消息内容"
当查询多个Talker时返回格式为"昵称(ID)\n[TalkerName(Talker)] 时间\n消息内容"
重要提示:
1. 当用户询问特定时间段内的聊天记录时,必须使用正确的时间格式,特别是包含小时和分钟的查询
2. 对于"今天下午4点到5点聊了啥"这类查询,正确的时间参数格式应为"2023-04-18/16:00~2023-04-18/17:00"
3. 当用户询问具体群聊中某人的聊天记录时,使用"sender"参数
4. 当用户询问包含特定关键词的聊天记录时,使用"keyword"参数`,
InputSchema: mcp.ToolSchema{
Type: "object",
Properties: mcp.M{
"time": mcp.M{
"type": "string",
"description": "查询的时间点或时间段。可以是具体时间,例如 YYYY-MM-DD也可以是时间段例如 YYYY-MM-DD~YYYY-MM-DD时间段之间用\"~\"分隔。",
"description": `指定查询的时间点或时间范围,格式必须严格遵循以下规则:
【单一时间点格式】
- 精确到日:"2023-04-18"或"20230418"
- 精确到分钟(必须包含斜杠和冒号):"2023-04-18/14:30"或"20230418/14:30"表示2023年4月18日14点30分
【时间范围格式】(使用"~"分隔起止时间)
- 日期范围:"2023-04-01~2023-04-18"
- 同一天的时间段:"2023-04-18/14:30~2023-04-18/15:45"
* 表示2023年4月18日14点30分到15点45分之间
【重要提示】包含小时分钟的格式必须使用斜杠和冒号:"/"和":"
正确示例:"2023-04-18/16:30"4月18日下午4点30分
错误示例:"2023-04-18 16:30"、"2023-04-18T16:30"
【其他支持的格式】
- 年份:"2023"
- 月份:"2023-04"或"202304"`,
},
"talker": mcp.M{
"type": "string",
"description": "交谈对象可以是联系人或群聊。支持使用ID、昵称、备注名等进行查询。",
"description": `指定对话方(联系人或群组)
- 可使用ID、昵称或备注名
- 多个对话方用","分隔,如:"张三,李四,工作群"
- 【重要】这是多步查询中唯一应保留的参数`,
},
"sender": mcp.M{
"type": "string",
"description": `指定群聊中的发送者
- 仅在查询群聊记录时有效
- 多个发送者用","分隔,如:"张三,李四"
- 可使用ID、昵称或备注名
【重要】查询特定发送者的消息时:
1. 第一步使用sender参数初步定位多个相关消息时间点
2. 后续步骤必须移除sender参数分别查询每个时间点前后的完整对话
3. 错误示例:对所有找到的消息一次性查询大范围上下文
4. 正确示例对每个时间点T分别执行查询"T前后15-30分钟"不带sender`,
},
"keyword": mcp.M{
"type": "string",
"description": `搜索内容中的关键词
- 支持正则表达式匹配
- 【重要】查询特定话题时:
1. 第一步使用keyword参数初步定位多个相关消息时间点
2. 后续步骤必须移除keyword参数分别查询每个时间点前后的完整对话
3. 错误示例:对所有找到的关键词消息一次性查询大范围上下文
4. 正确示例对每个时间点T分别执行查询"T前后15-30分钟"不带keyword`,
},
},
Required: []string{"time", "talker"},
},
}
ToolCurrentTime = mcp.Tool{
Name: "current_time",
Description: `获取当前系统时间返回RFC3339格式的时间字符串包含用户本地时区信息
使用场景:
- 当用户询问"总结今日聊天记录"、"本周都聊了啥"等当前时间问题
- 当用户提及"昨天"、"上周"、"本月"等相对时间概念,需要确定基准时间点
- 需要执行依赖当前时间的计算(如"上个月5号我们有开会吗"
返回示例2025-04-18T21:29:00+08:00
注意:此工具不需要任何输入参数,直接调用即可获取当前时间。`,
InputSchema: mcp.ToolSchema{
Type: "object",
Properties: mcp.M{},
},
}
ResourceRecentChat = mcp.Resource{
Name: "最近会话",
URI: "session://recent",

View File

@@ -7,6 +7,7 @@ import (
"fmt"
"net/url"
"strings"
"time"
"github.com/sjzar/chatlog/internal/chatlog/ctx"
"github.com/sjzar/chatlog/internal/chatlog/database"
@@ -83,6 +84,7 @@ func (s *Service) processMCP(session *mcp.Session, req *mcp.Request) {
ToolChatRoom,
ToolRecentChat,
ToolChatLog,
ToolCurrentTime,
}})
case mcp.MethodToolsCall:
err = s.toolsCall(session, req)
@@ -130,13 +132,13 @@ func (s *Service) toolsCall(session *mcp.Session, req *mcp.Request) error {
buf := &bytes.Buffer{}
switch callReq.Name {
case "query_contact":
query := ""
if v, ok := callReq.Arguments["query"]; ok {
query = v.(string)
keyword := ""
if v, ok := callReq.Arguments["keyword"]; ok {
keyword = v.(string)
}
limit := util.MustAnyToInt(callReq.Arguments["limit"])
offset := util.MustAnyToInt(callReq.Arguments["offset"])
list, err := s.db.GetContacts(query, limit, offset)
list, err := s.db.GetContacts(keyword, limit, offset)
if err != nil {
return fmt.Errorf("无法获取联系人列表: %v", err)
}
@@ -145,13 +147,13 @@ func (s *Service) toolsCall(session *mcp.Session, req *mcp.Request) error {
buf.WriteString(fmt.Sprintf("%s,%s,%s,%s\n", contact.UserName, contact.Alias, contact.Remark, contact.NickName))
}
case "query_chat_room":
query := ""
if v, ok := callReq.Arguments["query"]; ok {
query = v.(string)
keyword := ""
if v, ok := callReq.Arguments["keyword"]; ok {
keyword = v.(string)
}
limit := util.MustAnyToInt(callReq.Arguments["limit"])
offset := util.MustAnyToInt(callReq.Arguments["offset"])
list, err := s.db.GetChatRooms(query, limit, offset)
list, err := s.db.GetChatRooms(keyword, limit, offset)
if err != nil {
return fmt.Errorf("无法获取群聊列表: %v", err)
}
@@ -160,13 +162,13 @@ func (s *Service) toolsCall(session *mcp.Session, req *mcp.Request) error {
buf.WriteString(fmt.Sprintf("%s,%s,%s,%s,%d\n", chatRoom.Name, chatRoom.Remark, chatRoom.NickName, chatRoom.Owner, len(chatRoom.Users)))
}
case "query_recent_chat":
query := ""
if v, ok := callReq.Arguments["query"]; ok {
query = v.(string)
keyword := ""
if v, ok := callReq.Arguments["keyword"]; ok {
keyword = v.(string)
}
limit := util.MustAnyToInt(callReq.Arguments["limit"])
offset := util.MustAnyToInt(callReq.Arguments["offset"])
data, err := s.db.GetSessions(query, limit, offset)
data, err := s.db.GetSessions(keyword, limit, offset)
if err != nil {
return fmt.Errorf("无法获取会话列表: %v", err)
}
@@ -190,16 +192,29 @@ func (s *Service) toolsCall(session *mcp.Session, req *mcp.Request) error {
if v, ok := callReq.Arguments["talker"]; ok {
talker = v.(string)
}
sender := ""
if v, ok := callReq.Arguments["sender"]; ok {
sender = v.(string)
}
keyword := ""
if v, ok := callReq.Arguments["keyword"]; ok {
keyword = v.(string)
}
limit := util.MustAnyToInt(callReq.Arguments["limit"])
offset := util.MustAnyToInt(callReq.Arguments["offset"])
messages, err := s.db.GetMessages(start, end, talker, limit, offset)
messages, err := s.db.GetMessages(start, end, talker, sender, keyword, limit, offset)
if err != nil {
return fmt.Errorf("无法获取聊天记录: %v", err)
}
if len(messages) == 0 {
buf.WriteString("未找到符合查询条件的聊天记录")
}
for _, m := range messages {
buf.WriteString(m.PlainText(len(talker) == 0, ""))
buf.WriteString(m.PlainText(strings.Contains(talker, ","), util.PerfectTimeFormat(start, end), ""))
buf.WriteString("\n")
}
case "current_time":
buf.WriteString(time.Now().Local().Format(time.RFC3339))
default:
return fmt.Errorf("未支持的工具: %s", callReq.Name)
}
@@ -228,7 +243,6 @@ func (s *Service) resourcesRead(session *mcp.Session, req *mcp.Request) error {
buf := &bytes.Buffer{}
switch u.Scheme {
case "contact":
list, err := s.db.GetContacts(u.Host, 0, 0)
if err != nil {
return fmt.Errorf("无法获取联系人列表: %v", err)
@@ -262,12 +276,15 @@ func (s *Service) resourcesRead(session *mcp.Session, req *mcp.Request) error {
}
limit := util.MustAnyToInt(u.Query().Get("limit"))
offset := util.MustAnyToInt(u.Query().Get("offset"))
messages, err := s.db.GetMessages(start, end, u.Host, limit, offset)
messages, err := s.db.GetMessages(start, end, u.Host, "", "", limit, offset)
if err != nil {
return fmt.Errorf("无法获取聊天记录: %v", err)
}
if len(messages) == 0 {
buf.WriteString("未找到符合查询条件的聊天记录")
}
for _, m := range messages {
buf.WriteString(m.PlainText(len(u.Host) == 0, ""))
buf.WriteString(m.PlainText(strings.Contains(u.Host, ","), util.PerfectTimeFormat(start, end), ""))
buf.WriteString("\n")
}
default:

View File

@@ -5,24 +5,38 @@ import (
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"time"
"github.com/fsnotify/fsnotify"
"github.com/rs/zerolog/log"
"github.com/sjzar/chatlog/internal/chatlog/ctx"
"github.com/sjzar/chatlog/internal/errors"
"github.com/sjzar/chatlog/internal/wechat"
"github.com/sjzar/chatlog/internal/wechat/decrypt"
"github.com/sjzar/chatlog/pkg/filemonitor"
"github.com/sjzar/chatlog/pkg/util"
)
"github.com/rs/zerolog/log"
var (
DebounceTime = 1 * time.Second
MaxWaitTime = 10 * time.Second
)
type Service struct {
ctx *ctx.Context
lastEvents map[string]time.Time
pendingActions map[string]bool
mutex sync.Mutex
fm *filemonitor.FileMonitor
}
func NewService(ctx *ctx.Context) *Service {
return &Service{
ctx: ctx,
lastEvents: make(map[string]time.Time),
pendingActions: make(map[string]bool),
}
}
@@ -46,90 +60,128 @@ func (s *Service) GetDataKey(info *wechat.Account) (string, error) {
return key, nil
}
// FindDBFiles finds all .db files in the specified directory
func (s *Service) FindDBFiles(rootDir string, recursive bool) ([]string, error) {
// Check if directory exists
info, err := os.Stat(rootDir)
func (s *Service) StartAutoDecrypt() error {
dbGroup, err := filemonitor.NewFileGroup("wechat", s.ctx.DataDir, `.*\.db$`, []string{"fts"})
if err != nil {
return nil, fmt.Errorf("cannot access directory %s: %w", rootDir, err)
return err
}
dbGroup.AddCallback(s.DecryptFileCallback)
if !info.IsDir() {
return nil, fmt.Errorf("%s is not a directory", rootDir)
s.fm = filemonitor.NewFileMonitor()
s.fm.AddGroup(dbGroup)
if err := s.fm.Start(); err != nil {
log.Debug().Err(err).Msg("failed to start file monitor")
return err
}
var dbFiles []string
// Define walk function
walkFunc := func(path string, info os.FileInfo, err error) error {
if err != nil {
// If a file or directory can't be accessed, log the error but continue
log.Err(err).Msgf("Warning: Cannot access %s", path)
return nil
}
// If it's a directory and not the root directory, and we're not recursively searching, skip it
if info.IsDir() && path != rootDir && !recursive {
return filepath.SkipDir
}
// Check if file extension is .db
if !info.IsDir() && strings.ToLower(filepath.Ext(path)) == ".db" {
dbFiles = append(dbFiles, path)
}
return nil
}
// Start traversal
err = filepath.Walk(rootDir, walkFunc)
if err != nil {
return nil, fmt.Errorf("error traversing directory: %w", err)
}
if len(dbFiles) == 0 {
return nil, fmt.Errorf("no .db files found")
}
return dbFiles, nil
}
func (s *Service) DecryptDBFiles(dataDir string, workDir string, key string, platform string, version int) error {
func (s *Service) StopAutoDecrypt() error {
if s.fm != nil {
if err := s.fm.Stop(); err != nil {
return err
}
}
s.fm = nil
return nil
}
ctx := context.Background()
func (s *Service) DecryptFileCallback(event fsnotify.Event) error {
if event.Op.Has(fsnotify.Chmod) || !event.Op.Has(fsnotify.Write) {
return nil
}
dbfiles, err := s.FindDBFiles(dataDir, true)
s.mutex.Lock()
s.lastEvents[event.Name] = time.Now()
if !s.pendingActions[event.Name] {
s.pendingActions[event.Name] = true
s.mutex.Unlock()
go s.waitAndProcess(event.Name)
} else {
s.mutex.Unlock()
}
return nil
}
func (s *Service) waitAndProcess(dbFile string) {
start := time.Now()
for {
time.Sleep(DebounceTime)
s.mutex.Lock()
lastEventTime := s.lastEvents[dbFile]
elapsed := time.Since(lastEventTime)
totalElapsed := time.Since(start)
if elapsed >= DebounceTime || totalElapsed >= MaxWaitTime {
s.pendingActions[dbFile] = false
s.mutex.Unlock()
log.Debug().Msgf("Processing file: %s", dbFile)
s.DecryptDBFile(dbFile)
return
}
s.mutex.Unlock()
}
}
func (s *Service) DecryptDBFile(dbFile string) error {
decryptor, err := decrypt.NewDecryptor(s.ctx.Platform, s.ctx.Version)
if err != nil {
return err
}
decryptor, err := decrypt.NewDecryptor(platform, version)
if err != nil {
return err
}
for _, dbfile := range dbfiles {
output := filepath.Join(workDir, dbfile[len(dataDir):])
output := filepath.Join(s.ctx.WorkDir, dbFile[len(s.ctx.DataDir):])
if err := util.PrepareDir(filepath.Dir(output)); err != nil {
return err
}
outputFile, err := os.Create(output)
outputTemp := output + ".tmp"
outputFile, err := os.Create(outputTemp)
if err != nil {
return fmt.Errorf("failed to create output file: %v", err)
}
defer outputFile.Close()
defer func() {
outputFile.Close()
if err := os.Rename(outputTemp, output); err != nil {
log.Debug().Err(err).Msgf("failed to rename %s to %s", outputTemp, output)
}
}()
if err := decryptor.Decrypt(ctx, dbfile, key, outputFile); err != nil {
log.Err(err).Msgf("failed to decrypt %s", dbfile)
if err := decryptor.Decrypt(context.Background(), dbFile, s.ctx.DataKey, outputFile); err != nil {
if err == errors.ErrAlreadyDecrypted {
if data, err := os.ReadFile(dbfile); err == nil {
if data, err := os.ReadFile(dbFile); err == nil {
outputFile.Write(data)
}
continue
return nil
}
log.Err(err).Msgf("failed to decrypt %s", dbFile)
return err
}
log.Debug().Msgf("Decrypted %s to %s", dbFile, output)
return nil
}
func (s *Service) DecryptDBFiles() error {
dbGroup, err := filemonitor.NewFileGroup("wechat", s.ctx.DataDir, `.*\.db$`, []string{"fts"})
if err != nil {
return err
}
dbFiles, err := dbGroup.List()
if err != nil {
return err
}
for _, dbFile := range dbFiles {
if err := s.DecryptDBFile(dbFile); err != nil {
log.Debug().Msgf("DecryptDBFile %s failed: %v", dbFile, err)
continue
// return err
}
}

View File

@@ -65,7 +65,7 @@ func Newf(cause error, code int, format string, args ...interface{}) *Error {
return &Error{
Message: fmt.Sprintf(format, args...),
Cause: cause,
Code: http.StatusInternalServerError,
Code: code,
}
}
@@ -112,7 +112,7 @@ func RootCause(err error) error {
func Err(c *gin.Context, err error) {
if appErr, ok := err.(*Error); ok {
c.JSON(appErr.Code, appErr)
c.JSON(appErr.Code, appErr.Error())
return
}

View File

@@ -2,6 +2,7 @@ package errors
import (
"net/http"
"runtime/debug"
"github.com/gin-gonic/gin"
"github.com/google/uuid"
@@ -50,7 +51,7 @@ func RecoveryMiddleware() gin.HandlerFunc {
}
// 记录错误日志
log.Err(err).Msg("PANIC RECOVERED")
log.Err(err).Msgf("PANIC RECOVERED\n%s", string(debug.Stack()))
// 返回 500 错误
c.JSON(http.StatusInternalServerError, err)

View File

@@ -60,3 +60,7 @@ func ContactNotFound(key string) *Error {
func InitCacheFailed(cause error) *Error {
return New(cause, http.StatusInternalServerError, "init cache failed").WithStack()
}
func FileGroupNotFound(name string) *Error {
return Newf(nil, http.StatusNotFound, "file group not found: %s", name).WithStack()
}

View File

@@ -47,33 +47,6 @@ type ContactV3 struct {
Remark string `json:"Remark"`
NickName string `json:"NickName"`
Reserved1 int `json:"Reserved1"` // 1 自己好友或自己加入的群聊; 0 群聊成员(非好友)
// EncryptUserName string `json:"EncryptUserName"`
// DelFlag int `json:"DelFlag"`
// Type int `json:"Type"`
// VerifyFlag int `json:"VerifyFlag"`
// Reserved2 int `json:"Reserved2"`
// Reserved3 string `json:"Reserved3"`
// Reserved4 string `json:"Reserved4"`
// LabelIDList string `json:"LabelIDList"`
// DomainList string `json:"DomainList"`
// ChatRoomType int `json:"ChatRoomType"`
// PYInitial string `json:"PYInitial"`
// QuanPin string `json:"QuanPin"`
// RemarkPYInitial string `json:"RemarkPYInitial"`
// RemarkQuanPin string `json:"RemarkQuanPin"`
// BigHeadImgUrl string `json:"BigHeadImgUrl"`
// SmallHeadImgUrl string `json:"SmallHeadImgUrl"`
// HeadImgMd5 string `json:"HeadImgMd5"`
// ChatRoomNotify int `json:"ChatRoomNotify"`
// Reserved5 int `json:"Reserved5"`
// Reserved6 string `json:"Reserved6"`
// Reserved7 string `json:"Reserved7"`
// ExtraBuf []byte `json:"ExtraBuf"`
// Reserved8 int `json:"Reserved8"`
// Reserved9 int `json:"Reserved9"`
// Reserved10 string `json:"Reserved10"`
// Reserved11 string `json:"Reserved11"`
}
func (c *ContactV3) Wrap() *Contact {

View File

@@ -40,33 +40,6 @@ type ContactDarwinV3 struct {
M_nsRemark string `json:"m_nsRemark"`
M_uiSex int `json:"m_uiSex"`
M_nsAliasName string `json:"m_nsAliasName"`
// M_uiConType int `json:"m_uiConType"`
// M_nsShortPY string `json:"m_nsShortPY"`
// M_nsRemarkPYFull string `json:"m_nsRemarkPYFull"`
// M_nsRemarkPYShort string `json:"m_nsRemarkPYShort"`
// M_uiCertificationFlag int `json:"m_uiCertificationFlag"`
// M_uiType int `json:"m_uiType"` // 本来想拿这个字段来区分是否是好友,但是数据比较乱,好在 darwin v3 Contact 表中没有群聊成员
// M_nsImgStatus string `json:"m_nsImgStatus"`
// M_uiImgKey int `json:"m_uiImgKey"`
// M_nsHeadImgUrl string `json:"m_nsHeadImgUrl"`
// M_nsHeadHDImgUrl string `json:"m_nsHeadHDImgUrl"`
// M_nsHeadHDMd5 string `json:"m_nsHeadHDMd5"`
// M_nsChatRoomMemList string `json:"m_nsChatRoomMemList"`
// M_nsChatRoomAdminList string `json:"m_nsChatRoomAdminList"`
// M_uiChatRoomStatus int `json:"m_uiChatRoomStatus"`
// M_nsChatRoomDesc string `json:"m_nsChatRoomDesc"`
// M_nsDraft string `json:"m_nsDraft"`
// M_nsBrandIconUrl string `json:"m_nsBrandIconUrl"`
// M_nsGoogleContactName string `json:"m_nsGoogleContactName"`
// M_nsEncodeUserName string `json:"m_nsEncodeUserName"`
// M_uiChatRoomVersion int `json:"m_uiChatRoomVersion"`
// M_uiChatRoomMaxCount int `json:"m_uiChatRoomMaxCount"`
// M_uiChatRoomType int `json:"m_uiChatRoomType"`
// M_patSuffix string `json:"m_patSuffix"`
// RichChatRoomDesc string `json:"richChatRoomDesc"`
// Packed_WCContactData string `json:"_packed_WCContactData"`
// OpenIMInfo string `json:"openIMInfo"`
}
func (c *ContactDarwinV3) Wrap() *Contact {

View File

@@ -30,25 +30,6 @@ type ContactV4 struct {
Remark string `json:"remark"`
NickName string `json:"nick_name"`
LocalType int `json:"local_type"` // 2 群聊; 3 群聊成员(非好友); 5,6 企业微信;
// ID int `json:"id"`
// EncryptUserName string `json:"encrypt_username"`
// Flag int `json:"flag"`
// DeleteFlag int `json:"delete_flag"`
// VerifyFlag int `json:"verify_flag"`
// RemarkQuanPin string `json:"remark_quan_pin"`
// RemarkPinYinInitial string `json:"remark_pin_yin_initial"`
// PinYinInitial string `json:"pin_yin_initial"`
// QuanPin string `json:"quan_pin"`
// BigHeadUrl string `json:"big_head_url"`
// SmallHeadUrl string `json:"small_head_url"`
// HeadImgMd5 string `json:"head_img_md5"`
// ChatRoomNotify int `json:"chat_room_notify"`
// IsInChatRoom int `json:"is_in_chat_room"`
// Description string `json:"description"`
// ExtraBuffer []byte `json:"extra_buffer"`
// ChatRoomType int `json:"chat_room_type"`
}
func (c *ContactV4) Wrap() *Contact {

View File

@@ -10,6 +10,7 @@ type Media struct {
Path string `json:"path"`
Name string `json:"name"`
Size int64 `json:"size"`
Data []byte `json:"data"` // for voice
ModifyTime int64 `json:"modifyTime"`
}

View File

@@ -283,9 +283,34 @@ type FinderMegaVideo struct {
}
type SysMsg struct {
SysMsgTemplate SysMsgTemplate `xml:"sysmsgtemplate"`
Type string `xml:"type,attr"`
DelChatRoomMember *DelChatRoomMember `xml:"delchatroommember,omitempty"`
SysMsgTemplate *SysMsgTemplate `xml:"sysmsgtemplate,omitempty"`
}
// 第一种消息类型:删除群成员/二维码邀请
type DelChatRoomMember struct {
Plain string `xml:"plain"`
Text string `xml:"text"`
Link QRLink `xml:"link"`
}
type QRLink struct {
Scene string `xml:"scene"`
Text string `xml:"text"`
MemberList QRMemberList `xml:"memberlist"`
QRCode string `xml:"qrcode"`
}
type QRMemberList struct {
Usernames []UsernameItem `xml:"username"`
}
type UsernameItem struct {
Value string `xml:",chardata"`
}
// 第二种消息类型:系统消息模板
type SysMsgTemplate struct {
ContentTemplate ContentTemplate `xml:"content_template"`
}
@@ -305,7 +330,8 @@ type Link struct {
Name string `xml:"name,attr"`
Type string `xml:"type,attr"`
MemberList MemberList `xml:"memberlist"`
Separator string `xml:"separator"`
Separator string `xml:"separator,omitempty"`
Title string `xml:"title,omitempty"`
}
type MemberList struct {
@@ -318,6 +344,24 @@ type Member struct {
}
func (s *SysMsg) String() string {
if s.Type == "delchatroommember" {
return s.DelChatRoomMemberString()
}
return s.SysMsgTemplateString()
}
func (s *SysMsg) DelChatRoomMemberString() string {
if s.DelChatRoomMember == nil {
return ""
}
return s.DelChatRoomMember.Plain
}
func (s *SysMsg) SysMsgTemplateString() string {
if s.SysMsgTemplate == nil {
return ""
}
template := s.SysMsgTemplate.ContentTemplate.Template
links := s.SysMsgTemplate.ContentTemplate.LinkList.Links
@@ -354,8 +398,12 @@ func (s *SysMsg) String() string {
// 可以根据需要添加其他链接类型的处理逻辑
default:
if link.Title != "" {
replacement = link.Title
} else {
replacement = ""
}
}
// 将占位符名称和替换内容存入映射
replacements["$"+link.Name+"$"] = replacement

View File

@@ -55,6 +55,8 @@ func (m *Message) ParseMediaInfo(data string) error {
if Debug {
m.SysMsg = &sysMsg
}
m.Sender = "系统消息"
m.SenderName = ""
m.Content = sysMsg.String()
return nil
}
@@ -181,20 +183,19 @@ func (m *Message) SetContent(key string, value interface{}) {
m.Contents[key] = value
}
func (m *Message) PlainText(showChatRoom bool, host string) string {
func (m *Message) PlainText(showChatRoom bool, timeFormat string, host string) string {
if timeFormat == "" {
timeFormat = "01-02 15:04:05"
}
m.SetContent("host", host)
buf := strings.Builder{}
sender := m.Sender
switch {
case m.Type == 10000:
sender = "系统消息"
case m.IsSelf:
if m.IsSelf {
sender = "我"
default:
sender = m.Sender
}
if m.SenderName != "" {
buf.WriteString(m.SenderName)
@@ -219,7 +220,7 @@ func (m *Message) PlainText(showChatRoom bool, host string) string {
buf.WriteString("] ")
}
buf.WriteString(m.Time.Format("2006-01-02 15:04:05"))
buf.WriteString(m.Time.Format(timeFormat))
buf.WriteString("\n")
buf.WriteString(m.PlainTextContent())
@@ -235,6 +236,9 @@ func (m *Message) PlainTextContent() string {
case 3:
return fmt.Sprintf("![图片](http://%s/image/%s)", m.Contents["host"], m.Contents["md5"])
case 34:
if voice, ok := m.Contents["voice"]; ok {
return fmt.Sprintf("[语音](http://%s/voice/%s)", m.Contents["host"], voice)
}
return "[语音]"
case 42:
return "[名片]"
@@ -262,7 +266,11 @@ func (m *Message) PlainTextContent() string {
if !ok {
return "[合并转发]"
}
return recordInfo.String("", m.Contents["host"].(string))
host := ""
if m.Contents["host"] != nil {
host = m.Contents["host"].(string)
}
return recordInfo.String("", host)
case 33, 36:
if m.Contents["title"] == "" {
return "[小程序]"
@@ -290,7 +298,11 @@ func (m *Message) PlainTextContent() string {
return "> [引用]\n" + m.Content
}
buf := strings.Builder{}
referContent := refer.PlainText(false, m.Contents["host"].(string))
host := ""
if m.Contents["host"] != nil {
host = m.Contents["host"].(string)
}
referContent := refer.PlainText(false, "", host)
for _, line := range strings.Split(referContent, "\n") {
if line == "" {
continue

View File

@@ -1,6 +1,7 @@
package model
import (
"fmt"
"path/filepath"
"strings"
"time"
@@ -39,6 +40,7 @@ import (
// BytesTrans BLOB
// )
type MessageV3 struct {
MsgSvrID int64 `json:"MsgSvrID"` // 消息 ID
Sequence int64 `json:"Sequence"` // 消息序号10位时间戳 + 3位序号
CreateTime int64 `json:"CreateTime"` // 消息创建时间10位时间戳
StrTalker string `json:"StrTalker"` // 聊天对象,微信 ID or 群 ID
@@ -77,6 +79,11 @@ func (m *MessageV3) Wrap() *Message {
_m.ParseMediaInfo(_m.Content)
// 语音消息
if _m.Type == 34 {
_m.Contents["voice"] = fmt.Sprint(m.MsgSvrID)
}
if len(m.BytesExtra) != 0 {
if bytesExtra := ParseBytesExtra(m.BytesExtra); bytesExtra != nil {
if _m.IsChatRoom {

View File

@@ -2,6 +2,7 @@ package model
import (
"bytes"
"fmt"
"strings"
"time"
@@ -31,6 +32,7 @@ import (
// )
type MessageV4 struct {
SortSeq int64 `json:"sort_seq"` // 消息序号10位时间戳 + 3位序号
ServerID int64 `json:"server_id"` // 消息 ID用于关联 voice
LocalType int64 `json:"local_type"` // 消息类型
UserName string `json:"user_name"` // 发送人,通过 Join Name2Id 表获得
CreateTime int64 `json:"create_time"` // 消息创建时间10位时间戳
@@ -74,6 +76,11 @@ func (m *MessageV4) Wrap(talker string) *Message {
_m.ParseMediaInfo(content)
// 语音消息
if _m.Type == 34 {
_m.Contents["voice"] = fmt.Sprint(m.ServerID)
}
if len(m.PackedInfoData) != 0 {
if packedInfo := ParsePackedInfo(m.PackedInfoData); packedInfo != nil {
// FIXME 尝试解决 v4 版本 xml 数据无法匹配到 hardlink 记录的问题

258
internal/ui/form/form.go Normal file
View File

@@ -0,0 +1,258 @@
package form
import (
"fmt"
"github.com/gdamore/tcell/v2"
"github.com/rivo/tview"
"github.com/sjzar/chatlog/internal/ui/style"
)
const (
// DialogPadding dialog inner padding.
DialogPadding = 3
// DialogHelpHeight dialog help text height.
DialogHelpHeight = 1
// DialogMinWidth dialog min width.
DialogMinWidth = 40
// FormHeightOffset form height offset for border.
FormHeightOffset = 3
// 额外的宽度补偿,类似于 submenu 的 cmdWidthOffset
formWidthOffset = 10
)
// Form is a modal form component with a title, form fields, and help text.
type Form struct {
*tview.Box
title string
layout *tview.Flex
form *tview.Form
helpText *tview.TextView
width int
height int
cancelHandler func()
fields []formField // 存储字段信息以便重新计算宽度
}
// formField 存储表单字段的信息
type formField struct {
label string
value string
fieldWidth int
}
// NewForm creates a new form with the given title.
func NewForm(title string) *Form {
f := &Form{
Box: tview.NewBox(),
title: title,
layout: tview.NewFlex().SetDirection(tview.FlexRow),
form: tview.NewForm(),
fields: make([]formField, 0),
}
// 设置表单样式
f.form.SetBorderPadding(1, 1, 1, 1)
f.form.SetBackgroundColor(style.DialogBgColor)
f.form.SetFieldBackgroundColor(style.BgColor)
f.form.SetFieldTextColor(style.FgColor)
f.form.SetButtonBackgroundColor(style.ButtonBgColor)
f.form.SetButtonTextColor(style.FgColor)
f.form.SetLabelColor(style.DialogFgColor)
f.form.SetButtonsAlign(tview.AlignCenter)
// 创建帮助文本
f.helpText = tview.NewTextView()
f.helpText.SetDynamicColors(true)
f.helpText.SetTextAlign(tview.AlignCenter)
f.helpText.SetTextColor(style.DialogFgColor)
f.helpText.SetBackgroundColor(style.DialogBgColor)
fmt.Fprintf(f.helpText,
"[%s::b]Tab[%s::b]: 导航 [%s::b]Enter[%s::b]: 选择 [%s::b]ESC[%s::b]: 返回",
style.GetColorHex(style.MenuBgColor), style.GetColorHex(style.PageHeaderFgColor),
style.GetColorHex(style.MenuBgColor), style.GetColorHex(style.PageHeaderFgColor),
style.GetColorHex(style.MenuBgColor), style.GetColorHex(style.PageHeaderFgColor),
)
// 创建布局
formLayout := tview.NewFlex().SetDirection(tview.FlexColumn)
formLayout.AddItem(EmptyBoxSpace(style.DialogBgColor), 1, 0, false)
formLayout.AddItem(f.form, 0, 1, true)
formLayout.AddItem(EmptyBoxSpace(style.DialogBgColor), 1, 0, false)
// 设置主布局
f.layout.SetTitle(fmt.Sprintf("[::b]%s", f.title))
f.layout.SetTitleColor(style.DialogFgColor)
f.layout.SetTitleAlign(tview.AlignCenter)
f.layout.SetBorder(true)
f.layout.SetBorderColor(style.DialogBorderColor)
f.layout.SetBackgroundColor(style.DialogBgColor)
// 添加表单区域
f.layout.AddItem(formLayout, 0, 1, true)
// 添加帮助文本区域
f.layout.AddItem(f.helpText, DialogHelpHeight, 0, false)
return f
}
// AddInputField adds an input field to the form.
func (f *Form) AddInputField(label, value string, fieldWidth int, accept func(textToCheck string, lastChar rune) bool, changed func(text string)) *Form {
// 存储字段信息
f.fields = append(f.fields, formField{
label: label,
value: value,
fieldWidth: fieldWidth,
})
// 添加输入字段到表单
f.form.AddInputField(label, value, fieldWidth, accept, changed)
// 更新表单尺寸
f.recalculateSize()
return f
}
// AddButton adds a button to the form.
func (f *Form) AddButton(label string, selected func()) *Form {
f.form.AddButton(label, selected)
// 更新表单尺寸
f.recalculateSize()
return f
}
// AddCheckbox adds a checkbox to the form.
func (f *Form) AddCheckbox(label string, checked bool, changed func(checked bool)) *Form {
f.form.AddCheckbox(label, checked, changed)
// 更新表单尺寸
f.recalculateSize()
return f
}
// SetCancelFunc sets the function to be called when the form is cancelled.
func (f *Form) SetCancelFunc(handler func()) *Form {
f.cancelHandler = handler
return f
}
// recalculateSize 重新计算表单尺寸
func (f *Form) recalculateSize() {
// 计算表单项数量
itemCount := f.form.GetFormItemCount()
// 计算高度 - 每个表单项占2行按钮区域至少占2行再加上边框和帮助文本
f.height = (itemCount * 2) + 2 + FormHeightOffset + DialogHelpHeight
// 计算宽度 - 类似于 submenu 的实现
maxLabelWidth := 0
maxValueWidth := 0
// 遍历所有字段,找出最长的标签和值
for _, field := range f.fields {
if len(field.label) > maxLabelWidth {
maxLabelWidth = len(field.label)
}
// 对于值,使用字段宽度和实际值长度中的较大者
valueWidth := field.fieldWidth
if len(field.value) > valueWidth {
valueWidth = len(field.value)
}
if valueWidth > maxValueWidth {
maxValueWidth = valueWidth
}
}
// 计算总宽度,类似于 submenu 的计算方式
f.width = maxLabelWidth + maxValueWidth + formWidthOffset
// 确保宽度不小于最小值
if f.width < DialogMinWidth {
f.width = DialogMinWidth
}
}
// Draw draws the form on the screen.
func (f *Form) Draw(screen tcell.Screen) {
// 在绘制前重新计算尺寸,确保尺寸是最新的
f.recalculateSize()
// 绘制
f.Box.DrawForSubclass(screen, f)
f.layout.Draw(screen)
}
// SetRect sets the position and size of the form.
func (f *Form) SetRect(x, y, width, height int) {
// 确保尺寸是最新的
f.recalculateSize()
// 类似于 submenu 的实现
ws := (width - f.width) / 2
hs := (height - f.height) / 2
// 确保不会超出屏幕
if f.width > width {
ws = 0
f.width = width - 1
}
if f.height > height {
hs = 0
f.height = height - 1
}
// 设置表单位置
f.Box.SetRect(x+ws, y+hs, f.width, f.height)
// 获取内部矩形并设置布局
x, y, width, height = f.Box.GetInnerRect()
f.layout.SetRect(x, y, width, height)
}
// Focus is called when this primitive receives focus.
func (f *Form) Focus(delegate func(p tview.Primitive)) {
// 确保表单获得焦点
if f.form != nil {
delegate(f.form)
} else {
// 如果表单为空则让Box获得焦点
delegate(f.Box)
}
}
// HasFocus returns whether or not this primitive has focus.
func (f *Form) HasFocus() bool {
return f.form.HasFocus()
}
// InputHandler returns the handler for this primitive.
func (f *Form) InputHandler() func(event *tcell.EventKey, setFocus func(p tview.Primitive)) {
return f.WrapInputHandler(func(event *tcell.EventKey, setFocus func(p tview.Primitive)) {
// ESC键处理
if event.Key() == tcell.KeyEscape && f.cancelHandler != nil {
f.cancelHandler()
return
}
// 将事件传递给表单
if handler := f.form.InputHandler(); handler != nil {
handler(event, setFocus)
}
})
}
// EmptyBoxSpace creates an empty box with the specified background color.
func EmptyBoxSpace(bgColor tcell.Color) *tview.Box {
box := tview.NewBox()
box.SetBackgroundColor(bgColor)
box.SetBorder(false)
return box
}

View File

@@ -15,13 +15,14 @@ const (
// InfoBarViewHeight info bar height.
const (
InfoBarViewHeight = 6
InfoBarViewHeight = 7
accountRow = 0
pidRow = 1
statusRow = 2
dataUsageRow = 3
workUsageRow = 4
httpServerRow = 5
statusRow = 1
platformRow = 2
sessionRow = 3
dataUsageRow = 4
workUsageRow = 5
httpServerRow = 6
// 列索引
labelCol1 = 0 // 第一列标签
@@ -43,7 +44,7 @@ func New() *InfoBar {
table := tview.NewTable()
headerColor := style.InfoBarItemFgColor
// Account 和 Version
// Account 和 PID
table.SetCell(
accountRow,
labelCol1,
@@ -54,26 +55,11 @@ func New() *InfoBar {
table.SetCell(
accountRow,
labelCol2,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "Version:")),
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "PID:")),
)
table.SetCell(accountRow, valueCol2, tview.NewTableCell(""))
// PID 和 ExePath 行
table.SetCell(
pidRow,
labelCol1,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "PID:")),
)
table.SetCell(pidRow, valueCol1, tview.NewTableCell(""))
table.SetCell(
pidRow,
labelCol2,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "ExePath:")),
)
table.SetCell(pidRow, valueCol2, tview.NewTableCell(""))
// Status 和 Key 行
// Status 和 ExePath 行
table.SetCell(
statusRow,
labelCol1,
@@ -84,10 +70,40 @@ func New() *InfoBar {
table.SetCell(
statusRow,
labelCol2,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "Data Key:")),
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "ExePath:")),
)
table.SetCell(statusRow, valueCol2, tview.NewTableCell(""))
// Platform 和 Version 行
table.SetCell(
platformRow,
labelCol1,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "Platform:")),
)
table.SetCell(platformRow, valueCol1, tview.NewTableCell(""))
table.SetCell(
platformRow,
labelCol2,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "Version:")),
)
table.SetCell(platformRow, valueCol2, tview.NewTableCell(""))
// Session 和 Data Key 行
table.SetCell(
sessionRow,
labelCol1,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "Session:")),
)
table.SetCell(sessionRow, valueCol1, tview.NewTableCell(""))
table.SetCell(
sessionRow,
labelCol2,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "Data Key:")),
)
table.SetCell(sessionRow, valueCol2, tview.NewTableCell(""))
// Data Usage 和 Data Dir 行
table.SetCell(
dataUsageRow,
@@ -126,6 +142,13 @@ func New() *InfoBar {
)
table.SetCell(httpServerRow, valueCol1, tview.NewTableCell(""))
table.SetCell(
httpServerRow,
labelCol2,
tview.NewTableCell(fmt.Sprintf(" [%s::]%s", headerColor, "Auto Decrypt:")),
)
table.SetCell(httpServerRow, valueCol2, tview.NewTableCell(""))
// infobar
infoBar := &InfoBar{
Box: tview.NewBox(),
@@ -141,17 +164,25 @@ func (info *InfoBar) UpdateAccount(account string) {
}
func (info *InfoBar) UpdateBasicInfo(pid int, version string, exePath string) {
info.table.GetCell(pidRow, valueCol1).SetText(fmt.Sprintf("%d", pid))
info.table.GetCell(pidRow, valueCol2).SetText(exePath)
info.table.GetCell(accountRow, valueCol2).SetText(version)
info.table.GetCell(accountRow, valueCol2).SetText(fmt.Sprintf("%d", pid))
info.table.GetCell(statusRow, valueCol2).SetText(exePath)
info.table.GetCell(platformRow, valueCol2).SetText(version)
}
func (info *InfoBar) UpdateStatus(status string) {
info.table.GetCell(statusRow, valueCol1).SetText(status)
}
func (info *InfoBar) UpdatePlatform(text string) {
info.table.GetCell(platformRow, valueCol1).SetText(text)
}
func (info *InfoBar) UpdateSession(text string) {
info.table.GetCell(sessionRow, valueCol1).SetText(text)
}
func (info *InfoBar) UpdateDataKey(key string) {
info.table.GetCell(statusRow, valueCol2).SetText(key)
info.table.GetCell(sessionRow, valueCol2).SetText(key)
}
func (info *InfoBar) UpdateDataUsageDir(dataUsage string, dataDir string) {
@@ -169,6 +200,11 @@ func (info *InfoBar) UpdateHTTPServer(server string) {
info.table.GetCell(httpServerRow, valueCol1).SetText(server)
}
// UpdateAutoDecrypt updates Auto Decrypt value.
func (info *InfoBar) UpdateAutoDecrypt(text string) {
info.table.GetCell(httpServerRow, valueCol2).SetText(text)
}
// Draw draws this primitive onto the screen.
func (info *InfoBar) Draw(screen tcell.Screen) {
info.Box.DrawForSubclass(screen, info)

View File

@@ -22,7 +22,7 @@ const (
var V3KeyPatterns = []KeyPatternInfo{
{
Pattern: []byte{0x72, 0x74, 0x72, 0x65, 0x65, 0x5f, 0x69, 0x33, 0x32},
Offset: 24,
Offsets: []int{24},
},
}
@@ -122,16 +122,73 @@ func (e *V3Extractor) findMemory(ctx context.Context, pid uint32, memoryChannel
return err
}
log.Debug().Msgf("Read memory region, size: %d bytes", len(memory))
totalSize := len(memory)
log.Debug().Msgf("Read memory region, size: %d bytes", totalSize)
// Send memory data to channel for processing
// If memory is small enough, process it as a single chunk
if totalSize <= MinChunkSize {
select {
case memoryChannel <- memory:
log.Debug().Msg("Memory region sent for analysis")
log.Debug().Msg("Memory sent as a single chunk for analysis")
case <-ctx.Done():
return ctx.Err()
}
return nil
}
chunkCount := MaxWorkers * ChunkMultiplier
// Calculate chunk size based on fixed chunk count
chunkSize := totalSize / chunkCount
if chunkSize < MinChunkSize {
// Reduce number of chunks if each would be too small
chunkCount = totalSize / MinChunkSize
if chunkCount == 0 {
chunkCount = 1
}
chunkSize = totalSize / chunkCount
}
// Process memory in chunks from end to beginning
for i := chunkCount - 1; i >= 0; i-- {
select {
case <-ctx.Done():
return ctx.Err()
default:
// Calculate start and end positions for this chunk
start := i * chunkSize
end := (i + 1) * chunkSize
// Ensure the last chunk includes all remaining memory
if i == chunkCount-1 {
end = totalSize
}
// Add overlap area to catch patterns at chunk boundaries
if i > 0 {
start -= ChunkOverlapBytes
if start < 0 {
start = 0
}
}
chunk := memory[start:end]
log.Debug().
Int("chunk_index", i+1).
Int("total_chunks", chunkCount).
Int("chunk_size", len(chunk)).
Int("start_offset", start).
Int("end_offset", end).
Msg("Processing memory chunk")
select {
case memoryChannel <- chunk:
case <-ctx.Done():
return ctx.Err()
}
}
}
return nil
}
@@ -173,25 +230,27 @@ func (e *V3Extractor) SearchKey(ctx context.Context, memory []byte) (string, boo
break // No more matches found
}
// Try each offset for this pattern
for _, offset := range keyPattern.Offsets {
// Check if we have enough space for the key
keyOffset := index + keyPattern.Offset
keyOffset := index + offset
if keyOffset < 0 || keyOffset+32 > len(memory) {
index -= 1
continue
}
// Extract the key data, which is 32 bytes long
// Extract the key data, which is at the offset position and 32 bytes long
keyData := memory[keyOffset : keyOffset+32]
// Validate key against database header
if e.validator.Validate(keyData) {
log.Debug().
Str("pattern", hex.EncodeToString(keyPattern.Pattern)).
Int("offset", keyPattern.Offset).
Int("offset", offset).
Str("key", hex.EncodeToString(keyData)).
Msg("Key found")
return hex.EncodeToString(keyData), true
}
}
index -= 1
}

View File

@@ -17,16 +17,15 @@ import (
const (
MaxWorkers = 8
MinChunkSize = 1 * 1024 * 1024 // 1MB
ChunkOverlapBytes = 1024 // Greater than all offsets
ChunkMultiplier = 2 // Number of chunks = MaxWorkers * ChunkMultiplier
)
var V4KeyPatterns = []KeyPatternInfo{
{
Pattern: []byte{0x20, 0x66, 0x74, 0x73, 0x35, 0x28, 0x25, 0x00},
Offset: 16,
},
{
Pattern: []byte{0x20, 0x66, 0x74, 0x73, 0x35, 0x28, 0x25, 0x00},
Offset: -80,
Offsets: []int{16, -80, 64},
},
}
@@ -126,15 +125,73 @@ func (e *V4Extractor) findMemory(ctx context.Context, pid uint32, memoryChannel
return err
}
log.Debug().Msgf("Read memory region, size: %d bytes", len(memory))
totalSize := len(memory)
log.Debug().Msgf("Read memory region, size: %d bytes", totalSize)
// Send memory data to channel for processing
// If memory is small enough, process it as a single chunk
if totalSize <= MinChunkSize {
select {
case memoryChannel <- memory:
log.Debug().Msg("Memory region sent for analysis")
log.Debug().Msg("Memory sent as a single chunk for analysis")
case <-ctx.Done():
return ctx.Err()
}
return nil
}
chunkCount := MaxWorkers * ChunkMultiplier
// Calculate chunk size based on fixed chunk count
chunkSize := totalSize / chunkCount
if chunkSize < MinChunkSize {
// Reduce number of chunks if each would be too small
chunkCount = totalSize / MinChunkSize
if chunkCount == 0 {
chunkCount = 1
}
chunkSize = totalSize / chunkCount
}
// Process memory in chunks from end to beginning
for i := chunkCount - 1; i >= 0; i-- {
select {
case <-ctx.Done():
return ctx.Err()
default:
// Calculate start and end positions for this chunk
start := i * chunkSize
end := (i + 1) * chunkSize
// Ensure the last chunk includes all remaining memory
if i == chunkCount-1 {
end = totalSize
}
// Add overlap area to catch patterns at chunk boundaries
if i > 0 {
start -= ChunkOverlapBytes
if start < 0 {
start = 0
}
}
chunk := memory[start:end]
log.Debug().
Int("chunk_index", i+1).
Int("total_chunks", chunkCount).
Int("chunk_size", len(chunk)).
Int("start_offset", start).
Int("end_offset", end).
Msg("Processing memory chunk")
select {
case memoryChannel <- chunk:
case <-ctx.Done():
return ctx.Err()
}
}
}
return nil
}
@@ -177,25 +234,27 @@ func (e *V4Extractor) SearchKey(ctx context.Context, memory []byte) (string, boo
break // No more matches found
}
// Try each offset for this pattern
for _, offset := range keyPattern.Offsets {
// Check if we have enough space for the key
keyOffset := index + keyPattern.Offset
keyOffset := index + offset
if keyOffset < 0 || keyOffset+32 > len(memory) {
index -= 1
continue
}
// Extract the key data, which is 16 bytes after the pattern and 32 bytes long
// Extract the key data, which is at the offset position and 32 bytes long
keyData := memory[keyOffset : keyOffset+32]
// Validate key against database header
if e.validator.Validate(keyData) {
if keyData, ok := e.validate(ctx, keyData); ok {
log.Debug().
Str("pattern", hex.EncodeToString(keyPattern.Pattern)).
Int("offset", keyPattern.Offset).
Int("offset", offset).
Str("key", hex.EncodeToString(keyData)).
Msg("Key found")
return hex.EncodeToString(keyData), true
}
}
index -= 1
}
@@ -204,11 +263,19 @@ func (e *V4Extractor) SearchKey(ctx context.Context, memory []byte) (string, boo
return "", false
}
func (e *V4Extractor) validate(ctx context.Context, keyDate []byte) ([]byte, bool) {
if e.validator.Validate(keyDate) {
return keyDate, true
}
// Try to find a valid key by ***
return nil, false
}
func (e *V4Extractor) SetValidate(validator *decrypt.Validator) {
e.validator = validator
}
type KeyPatternInfo struct {
Pattern []byte
Offset int
Offsets []int
}

View File

@@ -13,8 +13,8 @@ import (
const (
V3ProcessName = "WeChat"
V4ProcessName = "Weixin"
V3DBFile = "Msg\\Misc.db"
V4DBFile = "db_storage\\message\\message_0.db"
V3DBFile = `Msg\Misc.db`
V4DBFile = `db_storage\session\session.db`
)
// Detector 实现 Windows 平台的进程检测器

View File

@@ -3,87 +3,134 @@ package darwinv3
import (
"context"
"crypto/md5"
"database/sql"
"encoding/hex"
"fmt"
"regexp"
"sort"
"strings"
"time"
"github.com/fsnotify/fsnotify"
_ "github.com/mattn/go-sqlite3"
"github.com/rs/zerolog/log"
"github.com/sjzar/chatlog/internal/errors"
"github.com/sjzar/chatlog/internal/model"
"github.com/sjzar/chatlog/internal/wechatdb/datasource/dbm"
"github.com/sjzar/chatlog/pkg/util"
)
const (
MessageFilePattern = "^msg_([0-9]?[0-9])?\\.db$"
ContactFilePattern = "^wccontact_new2\\.db$"
ChatRoomFilePattern = "^group_new\\.db$"
SessionFilePattern = "^session_new\\.db$"
MediaFilePattern = "^hldata\\.db$"
Message = "message"
Contact = "contact"
ChatRoom = "chatroom"
Session = "session"
Media = "media"
)
var Groups = []*dbm.Group{
{
Name: Message,
Pattern: `^msg_([0-9]?[0-9])?\.db$`,
BlackList: []string{},
},
{
Name: Contact,
Pattern: `^wccontact_new2\.db$`,
BlackList: []string{},
},
{
Name: ChatRoom,
Pattern: `group_new\.db$`,
BlackList: []string{},
},
{
Name: Session,
Pattern: `^session_new\.db$`,
BlackList: []string{},
},
{
Name: Media,
Pattern: `^hldata\.db$`,
BlackList: []string{},
},
}
type DataSource struct {
path string
messageDbs []*sql.DB
contactDb *sql.DB
chatRoomDb *sql.DB
sessionDb *sql.DB
mediaDb *sql.DB
dbm *dbm.DBManager
talkerDBMap map[string]*sql.DB
talkerDBMap map[string]string
user2DisplayName map[string]string
}
func New(path string) (*DataSource, error) {
ds := &DataSource{
path: path,
messageDbs: make([]*sql.DB, 0),
talkerDBMap: make(map[string]*sql.DB),
dbm: dbm.NewDBManager(path),
talkerDBMap: make(map[string]string),
user2DisplayName: make(map[string]string),
}
if err := ds.initMessageDbs(path); err != nil {
for _, g := range Groups {
ds.dbm.AddGroup(g)
}
if err := ds.dbm.Start(); err != nil {
return nil, err
}
if err := ds.initMessageDbs(); err != nil {
return nil, errors.DBInitFailed(err)
}
if err := ds.initContactDb(path); err != nil {
if err := ds.initChatRoomDb(); err != nil {
return nil, errors.DBInitFailed(err)
}
if err := ds.initChatRoomDb(path); err != nil {
return nil, errors.DBInitFailed(err)
ds.dbm.AddCallback(Message, func(event fsnotify.Event) error {
if !event.Op.Has(fsnotify.Create) {
return nil
}
if err := ds.initSessionDb(path); err != nil {
return nil, errors.DBInitFailed(err)
if err := ds.initMessageDbs(); err != nil {
log.Err(err).Msgf("Failed to reinitialize message DBs: %s", event.Name)
}
if err := ds.initMediaDb(path); err != nil {
return nil, errors.DBInitFailed(err)
return nil
})
ds.dbm.AddCallback(ChatRoom, func(event fsnotify.Event) error {
if !event.Op.Has(fsnotify.Create) {
return nil
}
if err := ds.initChatRoomDb(); err != nil {
log.Err(err).Msgf("Failed to reinitialize chatroom DB: %s", event.Name)
}
return nil
})
return ds, nil
}
func (ds *DataSource) initMessageDbs(path string) error {
func (ds *DataSource) SetCallback(name string, callback func(event fsnotify.Event) error) error {
return ds.dbm.AddCallback(name, callback)
}
files, err := util.FindFilesWithPatterns(path, MessageFilePattern, true)
func (ds *DataSource) initMessageDbs() error {
dbPaths, err := ds.dbm.GetDBPath(Message)
if err != nil {
return errors.DBFileNotFound(path, MessageFilePattern, err)
if strings.Contains(err.Error(), "db file not found") {
ds.talkerDBMap = make(map[string]string)
return nil
}
if len(files) == 0 {
return errors.DBFileNotFound(path, MessageFilePattern, nil)
return err
}
// 处理每个数据库文件
for _, filePath := range files {
// 连接数据库
db, err := sql.Open("sqlite3", filePath)
talkerDBMap := make(map[string]string)
for _, filePath := range dbPaths {
db, err := ds.dbm.OpenDB(filePath)
if err != nil {
log.Err(err).Msgf("连接数据库 %s 失败", filePath)
log.Err(err).Msgf("获取数据库 %s 失败", filePath)
continue
}
ds.messageDbs = append(ds.messageDbs, db)
// 获取所有表名
rows, err := db.Query("SELECT name FROM sqlite_master WHERE type='table' AND name LIKE 'Chat_%'")
@@ -104,110 +151,95 @@ func (ds *DataSource) initMessageDbs(path string) error {
if talkerMd5 == "" {
continue
}
ds.talkerDBMap[talkerMd5] = db
talkerDBMap[talkerMd5] = filePath
}
rows.Close()
}
ds.talkerDBMap = talkerDBMap
return nil
}
func (ds *DataSource) initContactDb(path string) error {
files, err := util.FindFilesWithPatterns(path, ContactFilePattern, true)
func (ds *DataSource) initChatRoomDb() error {
db, err := ds.dbm.GetDB(ChatRoom)
if err != nil {
return errors.DBFileNotFound(path, ContactFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, ContactFilePattern, nil)
}
ds.contactDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
if strings.Contains(err.Error(), "db file not found") {
ds.user2DisplayName = make(map[string]string)
return nil
}
func (ds *DataSource) initChatRoomDb(path string) error {
files, err := util.FindFilesWithPatterns(path, ChatRoomFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, ChatRoomFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, ChatRoomFilePattern, nil)
}
ds.chatRoomDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
return err
}
rows, err := ds.chatRoomDb.Query("SELECT m_nsUsrName, IFNULL(nickname,\"\") FROM GroupMember")
rows, err := db.Query("SELECT m_nsUsrName, IFNULL(nickname,\"\") FROM GroupMember")
if err != nil {
log.Err(err).Msgf("数据库 %s 获取群聊成员失败", files[0])
log.Err(err).Msg("获取群聊成员失败")
return nil
}
user2DisplayName := make(map[string]string)
for rows.Next() {
var user string
var nickName string
if err := rows.Scan(&user, &nickName); err != nil {
log.Err(err).Msgf("数据库 %s 扫描表名失败", files[0])
log.Err(err).Msg("扫描表名失败")
continue
}
ds.user2DisplayName[user] = nickName
user2DisplayName[user] = nickName
}
rows.Close()
ds.user2DisplayName = user2DisplayName
return nil
}
func (ds *DataSource) initSessionDb(path string) error {
files, err := util.FindFilesWithPatterns(path, SessionFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, SessionFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, SessionFilePattern, nil)
}
ds.sessionDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
return nil
}
func (ds *DataSource) initMediaDb(path string) error {
files, err := util.FindFilesWithPatterns(path, MediaFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, MediaFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, MediaFilePattern, nil)
}
ds.mediaDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
return nil
}
// GetMessages 实现获取消息的方法
func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, limit, offset int) ([]*model.Message, error) {
// 在 darwinv3 中,每个联系人/群聊的消息存储在单独的表中,表名为 Chat_md5(talker)
// 首先需要找到对应的表名
func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, sender string, keyword string, limit, offset int) ([]*model.Message, error) {
if talker == "" {
return nil, errors.ErrTalkerEmpty
}
_talkerMd5Bytes := md5.Sum([]byte(talker))
talkerMd5 := hex.EncodeToString(_talkerMd5Bytes[:])
db, ok := ds.talkerDBMap[talkerMd5]
if !ok {
return nil, errors.TalkerNotFound(talker)
// 解析talker参数支持多个talker以英文逗号分隔
talkers := util.Str2List(talker, ",")
if len(talkers) == 0 {
return nil, errors.ErrTalkerEmpty
}
// 解析sender参数支持多个发送者以英文逗号分隔
senders := util.Str2List(sender, ",")
// 预编译正则表达式如果有keyword
var regex *regexp.Regexp
if keyword != "" {
var err error
regex, err = regexp.Compile(keyword)
if err != nil {
return nil, errors.QueryFailed("invalid regex pattern", err)
}
}
// 从每个相关数据库中查询消息,并在读取时进行过滤
filteredMessages := []*model.Message{}
// 对每个talker进行查询
for _, talkerItem := range talkers {
// 检查上下文是否已取消
if err := ctx.Err(); err != nil {
return nil, err
}
// 在 darwinv3 中,需要先找到对应的数据库
_talkerMd5Bytes := md5.Sum([]byte(talkerItem))
talkerMd5 := hex.EncodeToString(_talkerMd5Bytes[:])
dbPath, ok := ds.talkerDBMap[talkerMd5]
if !ok {
// 如果找不到对应的数据库跳过此talker
continue
}
db, err := ds.dbm.OpenDB(dbPath)
if err != nil {
log.Error().Msgf("数据库 %s 未打开", dbPath)
continue
}
tableName := fmt.Sprintf("Chat_%s", talkerMd5)
// 构建查询条件
@@ -218,23 +250,18 @@ func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.T
ORDER BY msgCreateTime ASC
`, tableName)
if limit > 0 {
query += fmt.Sprintf(" LIMIT %d", limit)
if offset > 0 {
query += fmt.Sprintf(" OFFSET %d", offset)
}
}
// 执行查询
rows, err := db.QueryContext(ctx, query, startTime.Unix(), endTime.Unix())
if err != nil {
return nil, errors.QueryFailed(query, err)
// 如果表不存在跳过此talker
if strings.Contains(err.Error(), "no such table") {
continue
}
log.Err(err).Msgf("从数据库 %s 查询消息失败", dbPath)
continue
}
defer rows.Close()
// 处理查询结果
messages := []*model.Message{}
// 处理查询结果,在读取时进行过滤
for rows.Next() {
var msg model.MessageDarwinV3
err := rows.Scan(
@@ -244,16 +271,82 @@ func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.T
&msg.MesDes,
)
if err != nil {
rows.Close()
log.Err(err).Msgf("扫描消息行失败")
continue
}
// 将消息包装为通用模型
message := msg.Wrap(talker)
messages = append(messages, message)
message := msg.Wrap(talkerItem)
// 应用sender过滤
if len(senders) > 0 {
senderMatch := false
for _, s := range senders {
if message.Sender == s {
senderMatch = true
break
}
}
if !senderMatch {
continue // 不匹配sender跳过此消息
}
}
return messages, nil
// 应用keyword过滤
if regex != nil {
plainText := message.PlainTextContent()
if !regex.MatchString(plainText) {
continue // 不匹配keyword跳过此消息
}
}
// 通过所有过滤条件,保留此消息
filteredMessages = append(filteredMessages, message)
// 检查是否已经满足分页处理数量
if limit > 0 && len(filteredMessages) >= offset+limit {
// 已经获取了足够的消息,可以提前返回
rows.Close()
// 对所有消息按时间排序
sort.Slice(filteredMessages, func(i, j int) bool {
return filteredMessages[i].Seq < filteredMessages[j].Seq
})
// 处理分页
if offset >= len(filteredMessages) {
return []*model.Message{}, nil
}
end := offset + limit
if end > len(filteredMessages) {
end = len(filteredMessages)
}
return filteredMessages[offset:end], nil
}
}
rows.Close()
}
// 对所有消息按时间排序
// FIXME 不同 talker 需要使用 Time 排序
sort.Slice(filteredMessages, func(i, j int) bool {
return filteredMessages[i].Time.Before(filteredMessages[j].Time)
})
// 处理分页
if limit > 0 {
if offset >= len(filteredMessages) {
return []*model.Message{}, nil
}
end := offset + limit
if end > len(filteredMessages) {
end = len(filteredMessages)
}
return filteredMessages[offset:end], nil
}
return filteredMessages, nil
}
// 从表名中提取 talker
@@ -297,7 +390,11 @@ func (ds *DataSource) GetContacts(ctx context.Context, key string, limit, offset
}
// 执行查询
rows, err := ds.contactDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Contact)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -351,7 +448,11 @@ func (ds *DataSource) GetChatRooms(ctx context.Context, key string, limit, offse
}
// 执行查询
rows, err := ds.chatRoomDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(ChatRoom)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -380,7 +481,7 @@ func (ds *DataSource) GetChatRooms(ctx context.Context, key string, limit, offse
contacts, err := ds.GetContacts(ctx, key, 1, 0)
if err == nil && len(contacts) > 0 && strings.HasSuffix(contacts[0].UserName, "@chatroom") {
// 再次尝试通过用户名查找群聊
rows, err := ds.chatRoomDb.QueryContext(ctx,
rows, err := db.QueryContext(ctx,
`SELECT IFNULL(m_nsUsrName,""), IFNULL(nickname,""), IFNULL(m_nsRemark,""), IFNULL(m_nsChatRoomMemList,""), IFNULL(m_nsChatRoomAdminList,"")
FROM GroupContact
WHERE m_nsUsrName = ?`,
@@ -448,7 +549,11 @@ func (ds *DataSource) GetSessions(ctx context.Context, key string, limit, offset
}
// 执行查询
rows, err := ds.sessionDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Session)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -506,7 +611,11 @@ WHERE
r.mediaMd5 = ?`
args := []interface{}{key}
// 执行查询
rows, err := ds.mediaDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Media)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -541,46 +650,5 @@ WHERE
// Close 实现关闭数据库连接的方法
func (ds *DataSource) Close() error {
var errs []error
// 关闭消息数据库连接
for _, db := range ds.messageDbs {
if err := db.Close(); err != nil {
errs = append(errs, err)
}
}
// 关闭联系人数据库连接
if ds.contactDb != nil {
if err := ds.contactDb.Close(); err != nil {
errs = append(errs, err)
}
}
// 关闭群聊数据库连接
if ds.chatRoomDb != nil {
if err := ds.chatRoomDb.Close(); err != nil {
errs = append(errs, err)
}
}
// 关闭会话数据库连接
if ds.sessionDb != nil {
if err := ds.sessionDb.Close(); err != nil {
errs = append(errs, err)
}
}
// 关闭媒体数据库连接
if ds.mediaDb != nil {
if err := ds.mediaDb.Close(); err != nil {
errs = append(errs, err)
}
}
if len(errs) > 0 {
return errors.DBCloseFailed(errs[0])
}
return nil
return ds.dbm.Close()
}

View File

@@ -4,6 +4,8 @@ import (
"context"
"time"
"github.com/fsnotify/fsnotify"
"github.com/sjzar/chatlog/internal/errors"
"github.com/sjzar/chatlog/internal/model"
"github.com/sjzar/chatlog/internal/wechatdb/datasource/darwinv3"
@@ -14,7 +16,7 @@ import (
type DataSource interface {
// 消息
GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, limit, offset int) ([]*model.Message, error)
GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, sender string, keyword string, limit, offset int) ([]*model.Message, error)
// 联系人
GetContacts(ctx context.Context, key string, limit, offset int) ([]*model.Contact, error)
@@ -28,6 +30,9 @@ type DataSource interface {
// 媒体
GetMedia(ctx context.Context, _type string, key string) (*model.Media, error)
// 设置回调函数
SetCallback(name string, callback func(event fsnotify.Event) error) error
Close() error
}

View File

@@ -0,0 +1,170 @@
package dbm
import (
"database/sql"
"runtime"
"sync"
"time"
"github.com/fsnotify/fsnotify"
_ "github.com/mattn/go-sqlite3"
"github.com/rs/zerolog/log"
"github.com/sjzar/chatlog/internal/errors"
"github.com/sjzar/chatlog/pkg/filecopy"
"github.com/sjzar/chatlog/pkg/filemonitor"
)
type DBManager struct {
path string
fm *filemonitor.FileMonitor
fgs map[string]*filemonitor.FileGroup
dbs map[string]*sql.DB
dbPaths map[string][]string
mutex sync.RWMutex
}
func NewDBManager(path string) *DBManager {
return &DBManager{
path: path,
fm: filemonitor.NewFileMonitor(),
fgs: make(map[string]*filemonitor.FileGroup),
dbs: make(map[string]*sql.DB),
dbPaths: make(map[string][]string),
}
}
func (d *DBManager) AddGroup(g *Group) error {
fg, err := filemonitor.NewFileGroup(g.Name, d.path, g.Pattern, g.BlackList)
if err != nil {
return err
}
fg.AddCallback(d.Callback)
d.fm.AddGroup(fg)
d.mutex.Lock()
d.fgs[g.Name] = fg
d.mutex.Unlock()
return nil
}
func (d *DBManager) AddCallback(name string, callback func(event fsnotify.Event) error) error {
d.mutex.RLock()
fg, ok := d.fgs[name]
d.mutex.RUnlock()
if !ok {
return errors.FileGroupNotFound(name)
}
fg.AddCallback(callback)
return nil
}
func (d *DBManager) GetDB(name string) (*sql.DB, error) {
dbPaths, err := d.GetDBPath(name)
if err != nil {
return nil, err
}
return d.OpenDB(dbPaths[0])
}
func (d *DBManager) GetDBs(name string) ([]*sql.DB, error) {
dbPaths, err := d.GetDBPath(name)
if err != nil {
return nil, err
}
dbs := make([]*sql.DB, 0)
for _, file := range dbPaths {
db, err := d.OpenDB(file)
if err != nil {
return nil, err
}
dbs = append(dbs, db)
}
return dbs, nil
}
func (d *DBManager) GetDBPath(name string) ([]string, error) {
d.mutex.RLock()
dbPaths, ok := d.dbPaths[name]
d.mutex.RUnlock()
if !ok {
d.mutex.RLock()
fg, ok := d.fgs[name]
d.mutex.RUnlock()
if !ok {
return nil, errors.FileGroupNotFound(name)
}
list, err := fg.List()
if err != nil {
return nil, errors.DBFileNotFound(d.path, fg.PatternStr, err)
}
if len(list) == 0 {
return nil, errors.DBFileNotFound(d.path, fg.PatternStr, nil)
}
dbPaths = list
d.mutex.Lock()
d.dbPaths[name] = dbPaths
d.mutex.Unlock()
}
return dbPaths, nil
}
func (d *DBManager) OpenDB(path string) (*sql.DB, error) {
d.mutex.RLock()
db, ok := d.dbs[path]
d.mutex.RUnlock()
if ok {
return db, nil
}
var err error
tempPath := path
if runtime.GOOS == "windows" {
tempPath, err = filecopy.GetTempCopy(path)
if err != nil {
log.Err(err).Msgf("获取临时拷贝文件 %s 失败", path)
return nil, err
}
}
db, err = sql.Open("sqlite3", tempPath)
if err != nil {
log.Err(err).Msgf("连接数据库 %s 失败", path)
return nil, err
}
d.mutex.Lock()
d.dbs[path] = db
d.mutex.Unlock()
return db, nil
}
func (d *DBManager) Callback(event fsnotify.Event) error {
if !event.Op.Has(fsnotify.Create) {
return nil
}
d.mutex.Lock()
db, ok := d.dbs[event.Name]
if ok {
delete(d.dbs, event.Name)
go func(db *sql.DB) {
time.Sleep(time.Second * 5)
db.Close()
}(db)
}
d.mutex.Unlock()
return nil
}
func (d *DBManager) Start() error {
return d.fm.Start()
}
func (d *DBManager) Stop() error {
return d.fm.Stop()
}
func (d *DBManager) Close() error {
for _, db := range d.dbs {
db.Close()
}
return d.fm.Stop()
}

View File

@@ -0,0 +1,42 @@
package dbm
import (
"fmt"
"testing"
"time"
)
func TestXxx(t *testing.T) {
path := "/Users/sarv/Documents/chatlog/bigjun_9e7a"
g := &Group{
Name: "session",
Pattern: `session\.db$`,
BlackList: []string{},
}
d := NewDBManager(path)
d.AddGroup(g)
d.Start()
i := 0
for {
db, err := d.GetDB("session")
if err != nil {
fmt.Println(err)
break
}
var username string
row := db.QueryRow(`SELECT username FROM SessionTable LIMIT 1`)
if err := row.Scan(&username); err != nil {
fmt.Printf("Error scanning row: %v\n", err)
time.Sleep(100 * time.Millisecond)
continue
}
fmt.Printf("%d: Username: %s\n", i, username)
i++
time.Sleep(1000 * time.Millisecond)
}
}

View File

@@ -0,0 +1,7 @@
package dbm
type Group struct {
Name string
Pattern string
BlackList []string
}

View File

@@ -6,25 +6,57 @@ import (
"database/sql"
"encoding/hex"
"fmt"
"regexp"
"sort"
"strings"
"time"
"github.com/fsnotify/fsnotify"
_ "github.com/mattn/go-sqlite3"
"github.com/rs/zerolog/log"
"github.com/sjzar/chatlog/internal/errors"
"github.com/sjzar/chatlog/internal/model"
"github.com/sjzar/chatlog/internal/wechatdb/datasource/dbm"
"github.com/sjzar/chatlog/pkg/util"
)
const (
MessageFilePattern = "^message_([0-9]?[0-9])?\\.db$"
ContactFilePattern = "^contact\\.db$"
SessionFilePattern = "^session\\.db$"
MediaFilePattern = "^hardlink\\.db$"
Message = "message"
Contact = "contact"
Session = "session"
Media = "media"
Voice = "voice"
)
var Groups = []*dbm.Group{
{
Name: Message,
Pattern: `^message_([0-9]?[0-9])?\.db$`,
BlackList: []string{},
},
{
Name: Contact,
Pattern: `^contact\.db$`,
BlackList: []string{},
},
{
Name: Session,
Pattern: `session\.db$`,
BlackList: []string{},
},
{
Name: Media,
Pattern: `^hardlink\.db$`,
BlackList: []string{},
},
{
Name: Voice,
Pattern: `^media_([0-9]?[0-9])?\.db$`,
BlackList: []string{},
},
}
// MessageDBInfo 存储消息数据库的信息
type MessageDBInfo struct {
FilePath string
@@ -34,55 +66,68 @@ type MessageDBInfo struct {
type DataSource struct {
path string
messageDbs map[string]*sql.DB
contactDb *sql.DB
sessionDb *sql.DB
mediaDb *sql.DB
dbm *dbm.DBManager
// 消息数据库信息
messageFiles []MessageDBInfo
messageInfos []MessageDBInfo
}
func New(path string) (*DataSource, error) {
ds := &DataSource{
path: path,
messageDbs: make(map[string]*sql.DB),
messageFiles: make([]MessageDBInfo, 0),
dbm: dbm.NewDBManager(path),
messageInfos: make([]MessageDBInfo, 0),
}
if err := ds.initMessageDbs(path); err != nil {
for _, g := range Groups {
ds.dbm.AddGroup(g)
}
if err := ds.dbm.Start(); err != nil {
return nil, err
}
if err := ds.initMessageDbs(); err != nil {
return nil, errors.DBInitFailed(err)
}
if err := ds.initContactDb(path); err != nil {
return nil, errors.DBInitFailed(err)
ds.dbm.AddCallback(Message, func(event fsnotify.Event) error {
if !event.Op.Has(fsnotify.Create) {
return nil
}
if err := ds.initSessionDb(path); err != nil {
return nil, errors.DBInitFailed(err)
}
if err := ds.initMediaDb(path); err != nil {
return nil, errors.DBInitFailed(err)
if err := ds.initMessageDbs(); err != nil {
log.Err(err).Msgf("Failed to reinitialize message DBs: %s", event.Name)
}
return nil
})
return ds, nil
}
func (ds *DataSource) initMessageDbs(path string) error {
// 查找所有消息数据库文件
files, err := util.FindFilesWithPatterns(path, MessageFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, MessageFilePattern, err)
func (ds *DataSource) SetCallback(name string, callback func(event fsnotify.Event) error) error {
if name == "chatroom" {
name = Contact
}
return ds.dbm.AddCallback(name, callback)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, MessageFilePattern, nil)
func (ds *DataSource) initMessageDbs() error {
dbPaths, err := ds.dbm.GetDBPath(Message)
if err != nil {
if strings.Contains(err.Error(), "db file not found") {
ds.messageInfos = make([]MessageDBInfo, 0)
return nil
}
return err
}
// 处理每个数据库文件
for _, filePath := range files {
// 连接数据库
db, err := sql.Open("sqlite3", filePath)
infos := make([]MessageDBInfo, 0)
for _, filePath := range dbPaths {
db, err := ds.dbm.OpenDB(filePath)
if err != nil {
log.Err(err).Msgf("连接数据库 %s 失败", filePath)
log.Err(err).Msgf("获取数据库 %s 失败", filePath)
continue
}
@@ -93,90 +138,38 @@ func (ds *DataSource) initMessageDbs(path string) error {
row := db.QueryRow("SELECT timestamp FROM Timestamp LIMIT 1")
if err := row.Scan(&timestamp); err != nil {
log.Err(err).Msgf("获取数据库 %s 的时间戳失败", filePath)
db.Close()
continue
}
startTime = time.Unix(timestamp, 0)
// 保存数据库信息
ds.messageFiles = append(ds.messageFiles, MessageDBInfo{
infos = append(infos, MessageDBInfo{
FilePath: filePath,
StartTime: startTime,
})
// 保存数据库连接
ds.messageDbs[filePath] = db
}
// 按照 StartTime 排序数据库文件
sort.Slice(ds.messageFiles, func(i, j int) bool {
return ds.messageFiles[i].StartTime.Before(ds.messageFiles[j].StartTime)
sort.Slice(infos, func(i, j int) bool {
return infos[i].StartTime.Before(infos[j].StartTime)
})
// 设置结束时间
for i := range ds.messageFiles {
if i == len(ds.messageFiles)-1 {
ds.messageFiles[i].EndTime = time.Now()
for i := range infos {
if i == len(infos)-1 {
infos[i].EndTime = time.Now()
} else {
ds.messageFiles[i].EndTime = ds.messageFiles[i+1].StartTime
infos[i].EndTime = infos[i+1].StartTime
}
}
return nil
}
func (ds *DataSource) initContactDb(path string) error {
files, err := util.FindFilesWithPatterns(path, ContactFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, ContactFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, ContactFilePattern, nil)
}
ds.contactDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
return nil
}
func (ds *DataSource) initSessionDb(path string) error {
files, err := util.FindFilesWithPatterns(path, SessionFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, SessionFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, SessionFilePattern, nil)
}
ds.sessionDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
return nil
}
func (ds *DataSource) initMediaDb(path string) error {
files, err := util.FindFilesWithPatterns(path, MediaFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, MediaFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, MediaFilePattern, nil)
}
ds.mediaDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
ds.messageInfos = infos
return nil
}
// getDBInfosForTimeRange 获取时间范围内的数据库信息
func (ds *DataSource) getDBInfosForTimeRange(startTime, endTime time.Time) []MessageDBInfo {
var dbs []MessageDBInfo
for _, info := range ds.messageFiles {
for _, info := range ds.messageInfos {
if info.StartTime.Before(endTime) && info.EndTime.After(startTime) {
dbs = append(dbs, info)
}
@@ -184,24 +177,38 @@ func (ds *DataSource) getDBInfosForTimeRange(startTime, endTime time.Time) []Mes
return dbs
}
func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, limit, offset int) ([]*model.Message, error) {
func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, sender string, keyword string, limit, offset int) ([]*model.Message, error) {
if talker == "" {
return nil, errors.ErrTalkerEmpty
}
// 解析talker参数支持多个talker以英文逗号分隔
talkers := util.Str2List(talker, ",")
if len(talkers) == 0 {
return nil, errors.ErrTalkerEmpty
}
// 找到时间范围内的数据库文件
dbInfos := ds.getDBInfosForTimeRange(startTime, endTime)
if len(dbInfos) == 0 {
return nil, errors.TimeRangeNotFound(startTime, endTime)
}
if len(dbInfos) == 1 {
// LIMIT 和 OFFSET 逻辑在单文件情况下可以直接在 SQL 里处理
return ds.getMessagesSingleFile(ctx, dbInfos[0], startTime, endTime, talker, limit, offset)
// 解析sender参数支持多个发送者以英文逗号分隔
senders := util.Str2List(sender, ",")
// 预编译正则表达式如果有keyword
var regex *regexp.Regexp
if keyword != "" {
var err error
regex, err = regexp.Compile(keyword)
if err != nil {
return nil, errors.QueryFailed("invalid regex pattern", err)
}
}
// 从每个相关数据库中查询消息
totalMessages := []*model.Message{}
// 从每个相关数据库中查询消息,并在读取时进行过滤
filteredMessages := []*model.Message{}
for _, dbInfo := range dbInfos {
// 检查上下文是否已取消
@@ -209,124 +216,29 @@ func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.T
return nil, err
}
db, ok := ds.messageDbs[dbInfo.FilePath]
if !ok {
db, err := ds.dbm.OpenDB(dbInfo.FilePath)
if err != nil {
log.Error().Msgf("数据库 %s 未打开", dbInfo.FilePath)
continue
}
messages, err := ds.getMessagesFromDB(ctx, db, dbInfo, startTime, endTime, talker)
if err != nil {
log.Err(err).Msgf("从数据库 %s 获取消息失败", dbInfo.FilePath)
continue
}
totalMessages = append(totalMessages, messages...)
if limit+offset > 0 && len(totalMessages) >= limit+offset {
break
}
}
// 对所有消息按时间排序
sort.Slice(totalMessages, func(i, j int) bool {
return totalMessages[i].Seq < totalMessages[j].Seq
})
// 处理分页
if limit > 0 {
if offset >= len(totalMessages) {
return []*model.Message{}, nil
}
end := offset + limit
if end > len(totalMessages) {
end = len(totalMessages)
}
return totalMessages[offset:end], nil
}
return totalMessages, nil
}
// getMessagesSingleFile 从单个数据库文件获取消息
func (ds *DataSource) getMessagesSingleFile(ctx context.Context, dbInfo MessageDBInfo, startTime, endTime time.Time, talker string, limit, offset int) ([]*model.Message, error) {
db, ok := ds.messageDbs[dbInfo.FilePath]
if !ok {
return nil, errors.DBConnectFailed(dbInfo.FilePath, nil)
}
// 对每个talker进行查询
for _, talkerItem := range talkers {
// 构建表名
_talkerMd5Bytes := md5.Sum([]byte(talker))
talkerMd5 := hex.EncodeToString(_talkerMd5Bytes[:])
tableName := "Msg_" + talkerMd5
// 构建查询条件
conditions := []string{"create_time >= ? AND create_time <= ?"}
args := []interface{}{startTime.Unix(), endTime.Unix()}
query := fmt.Sprintf(`
SELECT m.sort_seq, m.local_type, n.user_name, m.create_time, m.message_content, m.packed_info_data, m.status
FROM %s m
LEFT JOIN Name2Id n ON m.real_sender_id = n.rowid
WHERE %s
ORDER BY m.sort_seq ASC
`, tableName, strings.Join(conditions, " AND "))
if limit > 0 {
query += fmt.Sprintf(" LIMIT %d", limit)
if offset > 0 {
query += fmt.Sprintf(" OFFSET %d", offset)
}
}
// 执行查询
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
defer rows.Close()
// 处理查询结果
messages := []*model.Message{}
for rows.Next() {
var msg model.MessageV4
err := rows.Scan(
&msg.SortSeq,
&msg.LocalType,
&msg.UserName,
&msg.CreateTime,
&msg.MessageContent,
&msg.PackedInfoData,
&msg.Status,
)
if err != nil {
return nil, errors.ScanRowFailed(err)
}
messages = append(messages, msg.Wrap(talker))
}
return messages, nil
}
// getMessagesFromDB 从数据库获取消息
func (ds *DataSource) getMessagesFromDB(ctx context.Context, db *sql.DB, dbInfo MessageDBInfo, startTime, endTime time.Time, talker string) ([]*model.Message, error) {
// 构建表名
_talkerMd5Bytes := md5.Sum([]byte(talker))
_talkerMd5Bytes := md5.Sum([]byte(talkerItem))
talkerMd5 := hex.EncodeToString(_talkerMd5Bytes[:])
tableName := "Msg_" + talkerMd5
// 检查表是否存在
var exists bool
err := db.QueryRowContext(ctx,
err = db.QueryRowContext(ctx,
"SELECT 1 FROM sqlite_master WHERE type='table' AND name=?",
tableName).Scan(&exists)
if err != nil {
if err == sql.ErrNoRows {
// 表不存在,返回空结果
return []*model.Message{}, nil
// 表不存在,继续下一个talker
continue
}
return nil, errors.QueryFailed("", err)
}
@@ -334,9 +246,11 @@ func (ds *DataSource) getMessagesFromDB(ctx context.Context, db *sql.DB, dbInfo
// 构建查询条件
conditions := []string{"create_time >= ? AND create_time <= ?"}
args := []interface{}{startTime.Unix(), endTime.Unix()}
log.Debug().Msgf("Table name: %s", tableName)
log.Debug().Msgf("Start time: %d, End time: %d", startTime.Unix(), endTime.Unix())
query := fmt.Sprintf(`
SELECT m.sort_seq, m.local_type, n.user_name, m.create_time, m.message_content, m.packed_info_data, m.status
SELECT m.sort_seq, m.server_id, m.local_type, n.user_name, m.create_time, m.message_content, m.packed_info_data, m.status
FROM %s m
LEFT JOIN Name2Id n ON m.real_sender_id = n.rowid
WHERE %s
@@ -348,19 +262,18 @@ func (ds *DataSource) getMessagesFromDB(ctx context.Context, db *sql.DB, dbInfo
if err != nil {
// 如果表不存在SQLite 会返回错误
if strings.Contains(err.Error(), "no such table") {
return []*model.Message{}, nil
continue
}
return nil, errors.QueryFailed(query, err)
log.Err(err).Msgf("从数据库 %s 查询消息失败", dbInfo.FilePath)
continue
}
defer rows.Close()
// 处理查询结果
messages := []*model.Message{}
// 处理查询结果,在读取时进行过滤
for rows.Next() {
var msg model.MessageV4
err := rows.Scan(
&msg.SortSeq,
&msg.ServerID,
&msg.LocalType,
&msg.UserName,
&msg.CreateTime,
@@ -369,13 +282,81 @@ func (ds *DataSource) getMessagesFromDB(ctx context.Context, db *sql.DB, dbInfo
&msg.Status,
)
if err != nil {
rows.Close()
return nil, errors.ScanRowFailed(err)
}
messages = append(messages, msg.Wrap(talker))
// 将消息转换为标准格式
message := msg.Wrap(talkerItem)
// 应用sender过滤
if len(senders) > 0 {
senderMatch := false
for _, s := range senders {
if message.Sender == s {
senderMatch = true
break
}
}
if !senderMatch {
continue // 不匹配sender跳过此消息
}
}
return messages, nil
// 应用keyword过滤
if regex != nil {
plainText := message.PlainTextContent()
if !regex.MatchString(plainText) {
continue // 不匹配keyword跳过此消息
}
}
// 通过所有过滤条件,保留此消息
filteredMessages = append(filteredMessages, message)
// 检查是否已经满足分页处理数量
if limit > 0 && len(filteredMessages) >= offset+limit {
// 已经获取了足够的消息,可以提前返回
rows.Close()
// 对所有消息按时间排序
sort.Slice(filteredMessages, func(i, j int) bool {
return filteredMessages[i].Seq < filteredMessages[j].Seq
})
// 处理分页
if offset >= len(filteredMessages) {
return []*model.Message{}, nil
}
end := offset + limit
if end > len(filteredMessages) {
end = len(filteredMessages)
}
return filteredMessages[offset:end], nil
}
}
rows.Close()
}
}
// 对所有消息按时间排序
sort.Slice(filteredMessages, func(i, j int) bool {
return filteredMessages[i].Seq < filteredMessages[j].Seq
})
// 处理分页
if limit > 0 {
if offset >= len(filteredMessages) {
return []*model.Message{}, nil
}
end := offset + limit
if end > len(filteredMessages) {
end = len(filteredMessages)
}
return filteredMessages[offset:end], nil
}
return filteredMessages, nil
}
// 联系人
@@ -404,7 +385,11 @@ func (ds *DataSource) GetContacts(ctx context.Context, key string, limit, offset
}
// 执行查询
rows, err := ds.contactDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Contact)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -436,13 +421,18 @@ func (ds *DataSource) GetChatRooms(ctx context.Context, key string, limit, offse
var query string
var args []interface{}
// 执行查询
db, err := ds.dbm.GetDB(Contact)
if err != nil {
return nil, err
}
if key != "" {
// 按照关键字查询
query = `SELECT username, owner, ext_buffer FROM chat_room WHERE username = ?`
args = []interface{}{key}
// 执行查询
rows, err := ds.contactDb.QueryContext(ctx, query, args...)
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -469,7 +459,7 @@ func (ds *DataSource) GetChatRooms(ctx context.Context, key string, limit, offse
contacts, err := ds.GetContacts(ctx, key, 1, 0)
if err == nil && len(contacts) > 0 && strings.HasSuffix(contacts[0].UserName, "@chatroom") {
// 再次尝试通过用户名查找群聊
rows, err := ds.contactDb.QueryContext(ctx,
rows, err := db.QueryContext(ctx,
`SELECT username, owner, ext_buffer FROM chat_room WHERE username = ?`,
contacts[0].UserName)
@@ -519,7 +509,7 @@ func (ds *DataSource) GetChatRooms(ctx context.Context, key string, limit, offse
}
// 执行查询
rows, err := ds.contactDb.QueryContext(ctx, query, args...)
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -573,7 +563,11 @@ func (ds *DataSource) GetSessions(ctx context.Context, key string, limit, offset
}
// 执行查询
rows, err := ds.sessionDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Session)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -605,10 +599,6 @@ func (ds *DataSource) GetMedia(ctx context.Context, _type string, key string) (*
return nil, errors.ErrKeyEmpty
}
if len(key) != 32 {
return nil, errors.ErrKeyLengthMust32
}
var table string
switch _type {
case "image":
@@ -617,6 +607,8 @@ func (ds *DataSource) GetMedia(ctx context.Context, _type string, key string) (*
table = "video_hardlink_info_v3"
case "file":
table = "file_hardlink_info_v3"
case "voice":
return ds.GetVoice(ctx, key)
default:
return nil, errors.MediaTypeUnsupported(_type)
}
@@ -639,7 +631,12 @@ func (ds *DataSource) GetMedia(ctx context.Context, _type string, key string) (*
query += " WHERE f.md5 = ? OR f.file_name LIKE ? || '%'"
args := []interface{}{key, key}
rows, err := ds.mediaDb.QueryContext(ctx, query, args...)
// 执行查询
db, err := ds.dbm.GetDB(Media)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -675,39 +672,51 @@ func (ds *DataSource) GetMedia(ctx context.Context, _type string, key string) (*
return media, nil
}
func (ds *DataSource) Close() error {
var errs []error
func (ds *DataSource) GetVoice(ctx context.Context, key string) (*model.Media, error) {
if key == "" {
return nil, errors.ErrKeyEmpty
}
// 关闭消息数据库连接
for _, db := range ds.messageDbs {
if err := db.Close(); err != nil {
errs = append(errs, err)
query := `
SELECT voice_data
FROM VoiceInfo
WHERE svr_id = ?
`
args := []interface{}{key}
dbs, err := ds.dbm.GetDBs(Voice)
if err != nil {
return nil, errors.DBConnectFailed("", err)
}
for _, db := range dbs {
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
defer rows.Close()
for rows.Next() {
var voiceData []byte
err := rows.Scan(
&voiceData,
)
if err != nil {
return nil, errors.ScanRowFailed(err)
}
if len(voiceData) > 0 {
return &model.Media{
Type: "voice",
Key: key,
Data: voiceData,
}, nil
}
}
}
// 关闭联系人数据库连接
if ds.contactDb != nil {
if err := ds.contactDb.Close(); err != nil {
errs = append(errs, err)
}
}
// 关闭会话数据库连接
if ds.sessionDb != nil {
if err := ds.sessionDb.Close(); err != nil {
errs = append(errs, err)
}
}
if ds.mediaDb != nil {
if err := ds.mediaDb.Close(); err != nil {
errs = append(errs, err)
}
}
if len(errs) > 0 {
return errors.DBCloseFailed(errs[0])
}
return nil
return nil, errors.ErrMediaNotFound
}
func (ds *DataSource) Close() error {
return ds.dbm.Close()
}

View File

@@ -2,29 +2,65 @@ package windowsv3
import (
"context"
"database/sql"
"encoding/hex"
"fmt"
"regexp"
"sort"
"strings"
"time"
"github.com/fsnotify/fsnotify"
_ "github.com/mattn/go-sqlite3"
"github.com/rs/zerolog/log"
"github.com/sjzar/chatlog/internal/errors"
"github.com/sjzar/chatlog/internal/model"
"github.com/sjzar/chatlog/internal/wechatdb/datasource/dbm"
"github.com/sjzar/chatlog/pkg/util"
)
const (
MessageFilePattern = "^MSG([0-9]?[0-9])?\\.db$"
ContactFilePattern = "^MicroMsg.db$"
ImageFilePattern = "^HardLinkImage\\.db$"
VideoFilePattern = "^HardLinkVideo\\.db$"
FileFilePattern = "^HardLinkFile\\.db$"
Message = "message"
Contact = "contact"
Image = "image"
Video = "video"
File = "file"
Voice = "voice"
)
var Groups = []*dbm.Group{
{
Name: Message,
Pattern: `^MSG([0-9]?[0-9])?\.db$`,
BlackList: []string{},
},
{
Name: Contact,
Pattern: `^MicroMsg\.db$`,
BlackList: []string{},
},
{
Name: Image,
Pattern: `^HardLinkImage\.db$`,
BlackList: []string{},
},
{
Name: Video,
Pattern: `^HardLinkVideo\.db$`,
BlackList: []string{},
},
{
Name: File,
Pattern: `^HardLinkFile\.db$`,
BlackList: []string{},
},
{
Name: Voice,
Pattern: `^MediaMSG([0-9])?\.db$`,
BlackList: []string{},
},
}
// MessageDBInfo 保存消息数据库的信息
type MessageDBInfo struct {
FilePath string
@@ -35,61 +71,71 @@ type MessageDBInfo struct {
// DataSource 实现了 DataSource 接口
type DataSource struct {
// 消息数据库
messageFiles []MessageDBInfo
messageDbs map[string]*sql.DB
path string
dbm *dbm.DBManager
// 联系人数据库
contactDbFile string
contactDb *sql.DB
imageDb *sql.DB
videoDb *sql.DB
fileDb *sql.DB
// 消息数据库信息
messageInfos []MessageDBInfo
}
// New 创建一个新的 WindowsV3DataSource
func New(path string) (*DataSource, error) {
ds := &DataSource{
messageFiles: make([]MessageDBInfo, 0),
messageDbs: make(map[string]*sql.DB),
path: path,
dbm: dbm.NewDBManager(path),
messageInfos: make([]MessageDBInfo, 0),
}
// 初始化消息数据库
if err := ds.initMessageDbs(path); err != nil {
for _, g := range Groups {
ds.dbm.AddGroup(g)
}
if err := ds.dbm.Start(); err != nil {
return nil, err
}
if err := ds.initMessageDbs(); err != nil {
return nil, errors.DBInitFailed(err)
}
// 初始化联系人数据库
if err := ds.initContactDb(path); err != nil {
return nil, errors.DBInitFailed(err)
ds.dbm.AddCallback(Message, func(event fsnotify.Event) error {
if !event.Op.Has(fsnotify.Create) {
return nil
}
if err := ds.initMediaDb(path); err != nil {
return nil, errors.DBInitFailed(err)
if err := ds.initMessageDbs(); err != nil {
log.Err(err).Msgf("Failed to reinitialize message DBs: %s", event.Name)
}
return nil
})
return ds, nil
}
// initMessageDbs 初始化消息数据库
func (ds *DataSource) initMessageDbs(path string) error {
// 查找所有消息数据库文件
files, err := util.FindFilesWithPatterns(path, MessageFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, MessageFilePattern, err)
func (ds *DataSource) SetCallback(name string, callback func(event fsnotify.Event) error) error {
if name == "chatroom" {
name = Contact
}
return ds.dbm.AddCallback(name, callback)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, MessageFilePattern, nil)
// initMessageDbs 初始化消息数据库
func (ds *DataSource) initMessageDbs() error {
// 获取所有消息数据库文件路径
dbPaths, err := ds.dbm.GetDBPath(Message)
if err != nil {
if strings.Contains(err.Error(), "db file not found") {
ds.messageInfos = make([]MessageDBInfo, 0)
return nil
}
return err
}
// 处理每个数据库文件
for _, filePath := range files {
// 连接数据库
db, err := sql.Open("sqlite3", filePath)
infos := make([]MessageDBInfo, 0)
for _, filePath := range dbPaths {
db, err := ds.dbm.OpenDB(filePath)
if err != nil {
log.Err(err).Msgf("连接数据库 %s 失败", filePath)
log.Err(err).Msgf("获取数据库 %s 失败", filePath)
continue
}
@@ -99,7 +145,6 @@ func (ds *DataSource) initMessageDbs(path string) error {
rows, err := db.Query("SELECT tableIndex, tableVersion, tableDesc FROM DBInfo")
if err != nil {
log.Err(err).Msgf("查询数据库 %s 的 DBInfo 表失败", filePath)
db.Close()
continue
}
@@ -126,7 +171,6 @@ func (ds *DataSource) initMessageDbs(path string) error {
rows, err = db.Query("SELECT UsrName FROM Name2ID")
if err != nil {
log.Err(err).Msgf("查询数据库 %s 的 Name2ID 表失败", filePath)
db.Close()
continue
}
@@ -143,105 +187,34 @@ func (ds *DataSource) initMessageDbs(path string) error {
rows.Close()
// 保存数据库信息
ds.messageFiles = append(ds.messageFiles, MessageDBInfo{
infos = append(infos, MessageDBInfo{
FilePath: filePath,
StartTime: startTime,
TalkerMap: talkerMap,
})
// 保存数据库连接
ds.messageDbs[filePath] = db
}
// 按照 StartTime 排序数据库文件
sort.Slice(ds.messageFiles, func(i, j int) bool {
return ds.messageFiles[i].StartTime.Before(ds.messageFiles[j].StartTime)
sort.Slice(infos, func(i, j int) bool {
return infos[i].StartTime.Before(infos[j].StartTime)
})
// 设置结束时间
for i := range ds.messageFiles {
if i == len(ds.messageFiles)-1 {
ds.messageFiles[i].EndTime = time.Now()
for i := range infos {
if i == len(infos)-1 {
infos[i].EndTime = time.Now()
} else {
ds.messageFiles[i].EndTime = ds.messageFiles[i+1].StartTime
infos[i].EndTime = infos[i+1].StartTime
}
}
return nil
}
// initContactDb 初始化联系人数据库
func (ds *DataSource) initContactDb(path string) error {
files, err := util.FindFilesWithPatterns(path, ContactFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, ContactFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, ContactFilePattern, nil)
}
ds.contactDbFile = files[0]
ds.contactDb, err = sql.Open("sqlite3", ds.contactDbFile)
if err != nil {
return errors.DBConnectFailed(ds.contactDbFile, err)
}
return nil
}
// initContactDb 初始化联系人数据库
func (ds *DataSource) initMediaDb(path string) error {
files, err := util.FindFilesWithPatterns(path, ImageFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, ImageFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, ImageFilePattern, nil)
}
ds.imageDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
files, err = util.FindFilesWithPatterns(path, VideoFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, VideoFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, VideoFilePattern, nil)
}
ds.videoDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
files, err = util.FindFilesWithPatterns(path, FileFilePattern, true)
if err != nil {
return errors.DBFileNotFound(path, FileFilePattern, err)
}
if len(files) == 0 {
return errors.DBFileNotFound(path, FileFilePattern, nil)
}
ds.fileDb, err = sql.Open("sqlite3", files[0])
if err != nil {
return errors.DBConnectFailed(files[0], err)
}
ds.messageInfos = infos
return nil
}
// getDBInfosForTimeRange 获取时间范围内的数据库信息
func (ds *DataSource) getDBInfosForTimeRange(startTime, endTime time.Time) []MessageDBInfo {
var dbs []MessageDBInfo
for _, info := range ds.messageFiles {
for _, info := range ds.messageInfos {
if info.StartTime.Before(endTime) && info.EndTime.After(startTime) {
dbs = append(dbs, info)
}
@@ -249,21 +222,38 @@ func (ds *DataSource) getDBInfosForTimeRange(startTime, endTime time.Time) []Mes
return dbs
}
// GetMessages 实现 DataSource 接口的 GetMessages 方法
func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, limit, offset int) ([]*model.Message, error) {
func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, sender string, keyword string, limit, offset int) ([]*model.Message, error) {
if talker == "" {
return nil, errors.ErrTalkerEmpty
}
// 解析talker参数支持多个talker以英文逗号分隔
talkers := util.Str2List(talker, ",")
if len(talkers) == 0 {
return nil, errors.ErrTalkerEmpty
}
// 找到时间范围内的数据库文件
dbInfos := ds.getDBInfosForTimeRange(startTime, endTime)
if len(dbInfos) == 0 {
return nil, errors.TimeRangeNotFound(startTime, endTime)
}
if len(dbInfos) == 1 {
// LIMIT 和 OFFSET 逻辑在单文件情况下可以直接在 SQL 里处理
return ds.getMessagesSingleFile(ctx, dbInfos[0], startTime, endTime, talker, limit, offset)
// 解析sender参数支持多个发送者以英文逗号分隔
senders := util.Str2List(sender, ",")
// 预编译正则表达式如果有keyword
var regex *regexp.Regexp
if keyword != "" {
var err error
regex, err = regexp.Compile(keyword)
if err != nil {
return nil, errors.QueryFailed("invalid regex pattern", err)
}
}
// 从每个相关数据库中查询消息
totalMessages := []*model.Message{}
filteredMessages := []*model.Message{}
for _, dbInfo := range dbInfos {
// 检查上下文是否已取消
@@ -271,29 +261,30 @@ func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.T
return nil, err
}
db, ok := ds.messageDbs[dbInfo.FilePath]
if !ok {
db, err := ds.dbm.OpenDB(dbInfo.FilePath)
if err != nil {
log.Error().Msgf("数据库 %s 未打开", dbInfo.FilePath)
continue
}
// 对每个talker进行查询
for _, talkerItem := range talkers {
// 构建查询条件
conditions := []string{"Sequence >= ? AND Sequence <= ?"}
args := []interface{}{startTime.Unix() * 1000, endTime.Unix() * 1000}
if len(talker) > 0 {
talkerID, ok := dbInfo.TalkerMap[talker]
// 添加talker条件
talkerID, ok := dbInfo.TalkerMap[talkerItem]
if ok {
conditions = append(conditions, "TalkerId = ?")
args = append(args, talkerID)
} else {
conditions = append(conditions, "StrTalker = ?")
args = append(args, talker)
}
args = append(args, talkerItem)
}
query := fmt.Sprintf(`
SELECT Sequence, CreateTime, StrTalker, IsSender,
SELECT MsgSvrID, Sequence, CreateTime, StrTalker, IsSender,
Type, SubType, StrContent, CompressContent, BytesExtra
FROM MSG
WHERE %s
@@ -303,17 +294,22 @@ func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.T
// 执行查询
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
log.Err(err).Msgf("查询数据库 %s 失败", dbInfo.FilePath)
// 如果表不存在跳过此talker
if strings.Contains(err.Error(), "no such table") {
continue
}
log.Err(err).Msgf("从数据库 %s 查询消息失败", dbInfo.FilePath)
continue
}
// 处理查询结果
// 处理查询结果,在读取时进行过滤
for rows.Next() {
var msg model.MessageV3
var compressContent []byte
var bytesExtra []byte
err := rows.Scan(
&msg.MsgSvrID,
&msg.Sequence,
&msg.CreateTime,
&msg.StrTalker,
@@ -325,105 +321,83 @@ func (ds *DataSource) GetMessages(ctx context.Context, startTime, endTime time.T
&bytesExtra,
)
if err != nil {
log.Err(err).Msg("扫描消息行失败")
continue
}
msg.CompressContent = compressContent
msg.BytesExtra = bytesExtra
totalMessages = append(totalMessages, msg.Wrap())
}
rows.Close()
if limit+offset > 0 && len(totalMessages) >= limit+offset {
break
}
}
// 对所有消息按时间排序
sort.Slice(totalMessages, func(i, j int) bool {
return totalMessages[i].Seq < totalMessages[j].Seq
})
// 处理分页
if limit > 0 {
if offset >= len(totalMessages) {
return []*model.Message{}, nil
}
end := offset + limit
if end > len(totalMessages) {
end = len(totalMessages)
}
return totalMessages[offset:end], nil
}
return totalMessages, nil
}
// getMessagesSingleFile 从单个数据库文件获取消息
func (ds *DataSource) getMessagesSingleFile(ctx context.Context, dbInfo MessageDBInfo, startTime, endTime time.Time, talker string, limit, offset int) ([]*model.Message, error) {
// 构建查询条件
conditions := []string{"Sequence >= ? AND Sequence <= ?"}
args := []interface{}{startTime.Unix() * 1000, endTime.Unix() * 1000}
if len(talker) > 0 {
// TalkerId 有索引,优先使用
talkerID, ok := dbInfo.TalkerMap[talker]
if ok {
conditions = append(conditions, "TalkerId = ?")
args = append(args, talkerID)
} else {
conditions = append(conditions, "StrTalker = ?")
args = append(args, talker)
}
}
query := fmt.Sprintf(`
SELECT Sequence, CreateTime, StrTalker, IsSender,
Type, SubType, StrContent, CompressContent, BytesExtra
FROM MSG
WHERE %s
ORDER BY Sequence ASC
`, strings.Join(conditions, " AND "))
if limit > 0 {
query += fmt.Sprintf(" LIMIT %d", limit)
if offset > 0 {
query += fmt.Sprintf(" OFFSET %d", offset)
}
}
// 执行查询
rows, err := ds.messageDbs[dbInfo.FilePath].QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
defer rows.Close()
// 处理查询结果
totalMessages := []*model.Message{}
for rows.Next() {
var msg model.MessageV3
var compressContent []byte
var bytesExtra []byte
err := rows.Scan(
&msg.Sequence,
&msg.CreateTime,
&msg.StrTalker,
&msg.IsSender,
&msg.Type,
&msg.SubType,
&msg.StrContent,
&compressContent,
&bytesExtra,
)
if err != nil {
return nil, errors.ScanRowFailed(err)
}
msg.CompressContent = compressContent
msg.BytesExtra = bytesExtra
totalMessages = append(totalMessages, msg.Wrap())
// 将消息转换为标准格式
message := msg.Wrap()
// 应用sender过滤
if len(senders) > 0 {
senderMatch := false
for _, s := range senders {
if message.Sender == s {
senderMatch = true
break
}
return totalMessages, nil
}
if !senderMatch {
continue // 不匹配sender跳过此消息
}
}
// 应用keyword过滤
if regex != nil {
plainText := message.PlainTextContent()
if !regex.MatchString(plainText) {
continue // 不匹配keyword跳过此消息
}
}
// 通过所有过滤条件,保留此消息
filteredMessages = append(filteredMessages, message)
// 检查是否已经满足分页处理数量
if limit > 0 && len(filteredMessages) >= offset+limit {
// 已经获取了足够的消息,可以提前返回
rows.Close()
// 对所有消息按时间排序
sort.Slice(filteredMessages, func(i, j int) bool {
return filteredMessages[i].Seq < filteredMessages[j].Seq
})
// 处理分页
if offset >= len(filteredMessages) {
return []*model.Message{}, nil
}
end := offset + limit
if end > len(filteredMessages) {
end = len(filteredMessages)
}
return filteredMessages[offset:end], nil
}
}
rows.Close()
}
}
// 对所有消息按时间排序
sort.Slice(filteredMessages, func(i, j int) bool {
return filteredMessages[i].Seq < filteredMessages[j].Seq
})
// 处理分页
if limit > 0 {
if offset >= len(filteredMessages) {
return []*model.Message{}, nil
}
end := offset + limit
if end > len(filteredMessages) {
end = len(filteredMessages)
}
return filteredMessages[offset:end], nil
}
return filteredMessages, nil
}
// GetContacts 实现获取联系人信息的方法
@@ -451,7 +425,11 @@ func (ds *DataSource) GetContacts(ctx context.Context, key string, limit, offset
}
// 执行查询
rows, err := ds.contactDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Contact)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -489,7 +467,11 @@ func (ds *DataSource) GetChatRooms(ctx context.Context, key string, limit, offse
args = []interface{}{key}
// 执行查询
rows, err := ds.contactDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Contact)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -516,7 +498,7 @@ func (ds *DataSource) GetChatRooms(ctx context.Context, key string, limit, offse
contacts, err := ds.GetContacts(ctx, key, 1, 0)
if err == nil && len(contacts) > 0 && strings.HasSuffix(contacts[0].UserName, "@chatroom") {
// 再次尝试通过用户名查找群聊
rows, err := ds.contactDb.QueryContext(ctx,
rows, err := db.QueryContext(ctx,
`SELECT ChatRoomName, Reserved2, RoomData FROM ChatRoom WHERE ChatRoomName = ?`,
contacts[0].UserName)
@@ -566,7 +548,11 @@ func (ds *DataSource) GetChatRooms(ctx context.Context, key string, limit, offse
}
// 执行查询
rows, err := ds.contactDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Contact)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -620,7 +606,11 @@ func (ds *DataSource) GetSessions(ctx context.Context, key string, limit, offset
}
// 执行查询
rows, err := ds.contactDb.QueryContext(ctx, query, args...)
db, err := ds.dbm.GetDB(Contact)
if err != nil {
return nil, err
}
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
@@ -652,30 +642,38 @@ func (ds *DataSource) GetMedia(ctx context.Context, _type string, key string) (*
return nil, errors.ErrKeyEmpty
}
if _type == "voice" {
return ds.GetVoice(ctx, key)
}
md5key, err := hex.DecodeString(key)
if err != nil {
return nil, errors.DecodeKeyFailed(err)
}
var db *sql.DB
var dbType string
var table1, table2 string
switch _type {
case "image":
db = ds.imageDb
dbType = Image
table1 = "HardLinkImageAttribute"
table2 = "HardLinkImageID"
case "video":
db = ds.videoDb
dbType = Video
table1 = "HardLinkVideoAttribute"
table2 = "HardLinkVideoID"
case "file":
db = ds.fileDb
dbType = File
table1 = "HardLinkFileAttribute"
table2 = "HardLinkFileID"
default:
return nil, errors.MediaTypeUnsupported(_type)
}
db, err := ds.dbm.GetDB(dbType)
if err != nil {
return nil, err
}
query := fmt.Sprintf(`
@@ -725,43 +723,52 @@ func (ds *DataSource) GetMedia(ctx context.Context, _type string, key string) (*
return media, nil
}
func (ds *DataSource) GetVoice(ctx context.Context, key string) (*model.Media, error) {
if key == "" {
return nil, errors.ErrKeyEmpty
}
query := `
SELECT Buf
FROM Media
WHERE Reserved0 = ?
`
args := []interface{}{key}
dbs, err := ds.dbm.GetDBs(Voice)
if err != nil {
return nil, errors.DBConnectFailed("", err)
}
for _, db := range dbs {
rows, err := db.QueryContext(ctx, query, args...)
if err != nil {
return nil, errors.QueryFailed(query, err)
}
defer rows.Close()
for rows.Next() {
var voiceData []byte
err := rows.Scan(
&voiceData,
)
if err != nil {
return nil, errors.ScanRowFailed(err)
}
if len(voiceData) > 0 {
return &model.Media{
Type: "voice",
Key: key,
Data: voiceData,
}, nil
}
}
}
return nil, errors.ErrMediaNotFound
}
// Close 实现 DataSource 接口的 Close 方法
func (ds *DataSource) Close() error {
var errs []error
// 关闭消息数据库连接
for _, db := range ds.messageDbs {
if err := db.Close(); err != nil {
errs = append(errs, err)
}
}
// 关闭联系人数据库连接
if ds.contactDb != nil {
if err := ds.contactDb.Close(); err != nil {
errs = append(errs, err)
}
}
if ds.imageDb != nil {
if err := ds.imageDb.Close(); err != nil {
errs = append(errs, err)
}
}
if ds.videoDb != nil {
if err := ds.videoDb.Close(); err != nil {
errs = append(errs, err)
}
}
if ds.fileDb != nil {
if err := ds.fileDb.Close(); err != nil {
errs = append(errs, err)
}
}
if len(errs) > 0 {
return errors.DBCloseFailed(errs[0])
}
return nil
return ds.dbm.Close()
}

View File

@@ -18,8 +18,8 @@ func (r *Repository) initChatRoomCache(ctx context.Context) error {
}
chatRoomMap := make(map[string]*model.ChatRoom)
remarkToChatRoom := make(map[string]*model.ChatRoom)
nickNameToChatRoom := make(map[string]*model.ChatRoom)
remarkToChatRoom := make(map[string][]*model.ChatRoom)
nickNameToChatRoom := make(map[string][]*model.ChatRoom)
chatRoomList := make([]string, 0)
chatRoomRemark := make([]string, 0)
chatRoomNickName := make([]string, 0)
@@ -30,11 +30,21 @@ func (r *Repository) initChatRoomCache(ctx context.Context) error {
chatRoomMap[chatRoom.Name] = chatRoom
chatRoomList = append(chatRoomList, chatRoom.Name)
if chatRoom.Remark != "" {
remarkToChatRoom[chatRoom.Remark] = chatRoom
remark, ok := remarkToChatRoom[chatRoom.Remark]
if !ok {
remark = make([]*model.ChatRoom, 0)
}
remark = append(remark, chatRoom)
remarkToChatRoom[chatRoom.Remark] = remark
chatRoomRemark = append(chatRoomRemark, chatRoom.Remark)
}
if chatRoom.NickName != "" {
nickNameToChatRoom[chatRoom.NickName] = chatRoom
nickName, ok := nickNameToChatRoom[chatRoom.NickName]
if !ok {
nickName = make([]*model.ChatRoom, 0)
}
nickName = append(nickName, chatRoom)
nickNameToChatRoom[chatRoom.NickName] = nickName
chatRoomNickName = append(chatRoomNickName, chatRoom.NickName)
}
}
@@ -49,11 +59,21 @@ func (r *Repository) initChatRoomCache(ctx context.Context) error {
chatRoomMap[contact.UserName] = chatRoom
chatRoomList = append(chatRoomList, contact.UserName)
if contact.Remark != "" {
remarkToChatRoom[contact.Remark] = chatRoom
remark, ok := remarkToChatRoom[chatRoom.Remark]
if !ok {
remark = make([]*model.ChatRoom, 0)
}
remark = append(remark, chatRoom)
remarkToChatRoom[chatRoom.Remark] = remark
chatRoomRemark = append(chatRoomRemark, contact.Remark)
}
if contact.NickName != "" {
nickNameToChatRoom[contact.NickName] = chatRoom
nickName, ok := nickNameToChatRoom[chatRoom.NickName]
if !ok {
nickName = make([]*model.ChatRoom, 0)
}
nickName = append(nickName, chatRoom)
nickNameToChatRoom[chatRoom.NickName] = nickName
chatRoomNickName = append(chatRoomNickName, contact.NickName)
}
}
@@ -63,9 +83,12 @@ func (r *Repository) initChatRoomCache(ctx context.Context) error {
sort.Strings(chatRoomNickName)
r.chatRoomCache = chatRoomMap
r.chatRoomList = chatRoomList
r.remarkToChatRoom = remarkToChatRoom
r.nickNameToChatRoom = nickNameToChatRoom
r.chatRoomList = chatRoomList
r.chatRoomRemark = chatRoomRemark
r.chatRoomNickName = chatRoomNickName
return nil
}
@@ -75,7 +98,7 @@ func (r *Repository) GetChatRooms(ctx context.Context, key string, limit, offset
if key != "" {
ret = r.findChatRooms(key)
if len(ret) == 0 {
return nil, errors.ChatRoomNotFound(key)
return []*model.ChatRoom{}, nil
}
if limit > 0 {
@@ -129,21 +152,21 @@ func (r *Repository) findChatRoom(key string) *model.ChatRoom {
return chatRoom
}
if chatRoom, ok := r.remarkToChatRoom[key]; ok {
return chatRoom
return chatRoom[0]
}
if chatRoom, ok := r.nickNameToChatRoom[key]; ok {
return chatRoom
return chatRoom[0]
}
// Contain
for _, remark := range r.chatRoomRemark {
if strings.Contains(remark, key) {
return r.remarkToChatRoom[remark]
return r.remarkToChatRoom[remark][0]
}
}
for _, nickName := range r.chatRoomNickName {
if strings.Contains(nickName, key) {
return r.nickNameToChatRoom[nickName]
return r.nickNameToChatRoom[nickName][0]
}
}
@@ -157,26 +180,42 @@ func (r *Repository) findChatRooms(key string) []*model.ChatRoom {
ret = append(ret, chatRoom)
distinct[chatRoom.Name] = true
}
if chatRoom, ok := r.remarkToChatRoom[key]; ok && !distinct[chatRoom.Name] {
if chatRooms, ok := r.remarkToChatRoom[key]; ok {
for _, chatRoom := range chatRooms {
if !distinct[chatRoom.Name] {
ret = append(ret, chatRoom)
distinct[chatRoom.Name] = true
}
if chatRoom, ok := r.nickNameToChatRoom[key]; ok && !distinct[chatRoom.Name] {
}
}
if chatRooms, ok := r.nickNameToChatRoom[key]; ok {
for _, chatRoom := range chatRooms {
if !distinct[chatRoom.Name] {
ret = append(ret, chatRoom)
distinct[chatRoom.Name] = true
}
}
}
// Contain
for _, remark := range r.chatRoomRemark {
if strings.Contains(remark, key) && !distinct[r.remarkToChatRoom[remark].Name] {
ret = append(ret, r.remarkToChatRoom[remark])
distinct[r.remarkToChatRoom[remark].Name] = true
if strings.Contains(remark, key) {
for _, chatRoom := range r.remarkToChatRoom[remark] {
if !distinct[chatRoom.Name] {
ret = append(ret, chatRoom)
distinct[chatRoom.Name] = true
}
}
}
}
for _, nickName := range r.chatRoomNickName {
if strings.Contains(nickName, key) && !distinct[r.nickNameToChatRoom[nickName].Name] {
ret = append(ret, r.nickNameToChatRoom[nickName])
distinct[r.nickNameToChatRoom[nickName].Name] = true
if strings.Contains(nickName, key) {
for _, chatRoom := range r.nickNameToChatRoom[nickName] {
if !distinct[chatRoom.Name] {
ret = append(ret, chatRoom)
distinct[chatRoom.Name] = true
}
}
}
}

View File

@@ -18,9 +18,9 @@ func (r *Repository) initContactCache(ctx context.Context) error {
}
contactMap := make(map[string]*model.Contact)
aliasMap := make(map[string]*model.Contact)
remarkMap := make(map[string]*model.Contact)
nickNameMap := make(map[string]*model.Contact)
aliasMap := make(map[string][]*model.Contact)
remarkMap := make(map[string][]*model.Contact)
nickNameMap := make(map[string][]*model.Contact)
chatRoomUserMap := make(map[string]*model.Contact)
chatRoomInContactMap := make(map[string]*model.Contact)
contactList := make([]string, 0)
@@ -34,15 +34,30 @@ func (r *Repository) initContactCache(ctx context.Context) error {
// 建立快速查找索引
if contact.Alias != "" {
aliasMap[contact.Alias] = contact
alias, ok := aliasMap[contact.Alias]
if !ok {
alias = make([]*model.Contact, 0)
}
alias = append(alias, contact)
aliasMap[contact.Alias] = alias
aliasList = append(aliasList, contact.Alias)
}
if contact.Remark != "" {
remarkMap[contact.Remark] = contact
remark, ok := remarkMap[contact.Remark]
if !ok {
remark = make([]*model.Contact, 0)
}
remark = append(remark, contact)
remarkMap[contact.Remark] = remark
remarkList = append(remarkList, contact.Remark)
}
if contact.NickName != "" {
nickNameMap[contact.NickName] = contact
nickName, ok := nickNameMap[contact.NickName]
if !ok {
nickName = make([]*model.Contact, 0)
}
nickName = append(nickName, contact)
nickNameMap[contact.NickName] = nickName
nickNameList = append(nickNameList, contact.NickName)
}
@@ -88,7 +103,7 @@ func (r *Repository) GetContacts(ctx context.Context, key string, limit, offset
if key != "" {
ret = r.findContacts(key)
if len(ret) == 0 {
return nil, errors.ContactNotFound(key)
return []*model.Contact{}, nil
}
if limit > 0 {
end := offset + limit
@@ -124,29 +139,29 @@ func (r *Repository) findContact(key string) *model.Contact {
return contact
}
if contact, ok := r.aliasToContact[key]; ok {
return contact
return contact[0]
}
if contact, ok := r.remarkToContact[key]; ok {
return contact
return contact[0]
}
if contact, ok := r.nickNameToContact[key]; ok {
return contact
return contact[0]
}
// Contain
for _, alias := range r.aliasList {
if strings.Contains(alias, key) {
return r.aliasToContact[alias]
return r.aliasToContact[alias][0]
}
}
for _, remark := range r.remarkList {
if strings.Contains(remark, key) {
return r.remarkToContact[remark]
return r.remarkToContact[remark][0]
}
}
for _, nickName := range r.nickNameList {
if strings.Contains(nickName, key) {
return r.nickNameToContact[nickName]
return r.nickNameToContact[nickName][0]
}
}
return nil
@@ -159,37 +174,62 @@ func (r *Repository) findContacts(key string) []*model.Contact {
ret = append(ret, contact)
distinct[contact.UserName] = true
}
if contact, ok := r.aliasToContact[key]; ok && !distinct[contact.UserName] {
if contacts, ok := r.aliasToContact[key]; ok {
for _, contact := range contacts {
if !distinct[contact.UserName] {
ret = append(ret, contact)
distinct[contact.UserName] = true
}
if contact, ok := r.remarkToContact[key]; ok && !distinct[contact.UserName] {
}
}
if contacts, ok := r.remarkToContact[key]; ok {
for _, contact := range contacts {
if !distinct[contact.UserName] {
ret = append(ret, contact)
distinct[contact.UserName] = true
}
if contact, ok := r.nickNameToContact[key]; ok && !distinct[contact.UserName] {
}
}
if contacts, ok := r.nickNameToContact[key]; ok {
for _, contact := range contacts {
if !distinct[contact.UserName] {
ret = append(ret, contact)
distinct[contact.UserName] = true
}
}
}
// Contain
for _, alias := range r.aliasList {
if strings.Contains(alias, key) && !distinct[r.aliasToContact[alias].UserName] {
ret = append(ret, r.aliasToContact[alias])
distinct[r.aliasToContact[alias].UserName] = true
if strings.Contains(alias, key) {
for _, contact := range r.aliasToContact[alias] {
if !distinct[contact.UserName] {
ret = append(ret, contact)
distinct[contact.UserName] = true
}
}
}
}
for _, remark := range r.remarkList {
if strings.Contains(remark, key) && !distinct[r.remarkToContact[remark].UserName] {
ret = append(ret, r.remarkToContact[remark])
distinct[r.remarkToContact[remark].UserName] = true
if strings.Contains(remark, key) {
for _, contact := range r.remarkToContact[remark] {
if !distinct[contact.UserName] {
ret = append(ret, contact)
distinct[contact.UserName] = true
}
}
}
}
for _, nickName := range r.nickNameList {
if strings.Contains(nickName, key) && !distinct[r.nickNameToContact[nickName].UserName] {
ret = append(ret, r.nickNameToContact[nickName])
distinct[r.nickNameToContact[nickName].UserName] = true
if strings.Contains(nickName, key) {
for _, contact := range r.nickNameToContact[nickName] {
if !distinct[contact.UserName] {
ret = append(ret, contact)
distinct[contact.UserName] = true
}
}
}
}
return ret
}

View File

@@ -2,23 +2,20 @@ package repository
import (
"context"
"strings"
"time"
"github.com/sjzar/chatlog/internal/model"
"github.com/sjzar/chatlog/pkg/util"
"github.com/rs/zerolog/log"
)
// GetMessages 实现 Repository 接口的 GetMessages 方法
func (r *Repository) GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, limit, offset int) ([]*model.Message, error) {
func (r *Repository) GetMessages(ctx context.Context, startTime, endTime time.Time, talker string, sender string, keyword string, limit, offset int) ([]*model.Message, error) {
if contact, _ := r.GetContact(ctx, talker); contact != nil {
talker = contact.UserName
} else if chatRoom, _ := r.GetChatRoom(ctx, talker); chatRoom != nil {
talker = chatRoom.Name
}
messages, err := r.ds.GetMessages(ctx, startTime, endTime, talker, limit, offset)
talker, sender = r.parseTalkerAndSender(ctx, talker, sender)
messages, err := r.ds.GetMessages(ctx, startTime, endTime, talker, sender, keyword, limit, offset)
if err != nil {
return nil, err
}
@@ -62,3 +59,53 @@ func (r *Repository) enrichMessage(msg *model.Message) {
}
}
}
func (r *Repository) parseTalkerAndSender(ctx context.Context, talker, sender string) (string, string) {
displayName2User := make(map[string]string)
users := make(map[string]bool)
talkers := util.Str2List(talker, ",")
if len(talkers) > 0 {
for i := 0; i < len(talkers); i++ {
if contact, _ := r.GetContact(ctx, talkers[i]); contact != nil {
talkers[i] = contact.UserName
} else if chatRoom, _ := r.GetChatRoom(ctx, talker); chatRoom != nil {
talkers[i] = chatRoom.Name
}
}
// 获取群聊的用户列表
for i := 0; i < len(talkers); i++ {
if chatRoom, _ := r.GetChatRoom(ctx, talkers[i]); chatRoom != nil {
for user, displayName := range chatRoom.User2DisplayName {
displayName2User[displayName] = user
}
for _, user := range chatRoom.Users {
users[user.UserName] = true
}
}
}
talker = strings.Join(talkers, ",")
}
senders := util.Str2List(sender, ",")
if len(senders) > 0 {
for i := 0; i < len(senders); i++ {
if user, ok := displayName2User[senders[i]]; ok {
senders[i] = user
} else {
// FIXME 大量群聊用户名称重复,无法直接通过 GetContact 获取 ID后续再优化
for user := range users {
if contact := r.getFullContact(user); contact != nil {
if contact.DisplayName() == senders[i] {
senders[i] = user
break
}
}
}
}
}
sender = strings.Join(senders, ",")
}
return talker, sender
}

View File

@@ -3,6 +3,9 @@ package repository
import (
"context"
"github.com/fsnotify/fsnotify"
"github.com/rs/zerolog/log"
"github.com/sjzar/chatlog/internal/errors"
"github.com/sjzar/chatlog/internal/model"
"github.com/sjzar/chatlog/internal/wechatdb/datasource"
@@ -14,9 +17,9 @@ type Repository struct {
// Cache for contact
contactCache map[string]*model.Contact
aliasToContact map[string]*model.Contact
remarkToContact map[string]*model.Contact
nickNameToContact map[string]*model.Contact
aliasToContact map[string][]*model.Contact
remarkToContact map[string][]*model.Contact
nickNameToContact map[string][]*model.Contact
chatRoomInContact map[string]*model.Contact
contactList []string
aliasList []string
@@ -25,8 +28,8 @@ type Repository struct {
// Cache for chat room
chatRoomCache map[string]*model.ChatRoom
remarkToChatRoom map[string]*model.ChatRoom
nickNameToChatRoom map[string]*model.ChatRoom
remarkToChatRoom map[string][]*model.ChatRoom
nickNameToChatRoom map[string][]*model.ChatRoom
chatRoomList []string
chatRoomRemark []string
chatRoomNickName []string
@@ -40,17 +43,17 @@ func New(ds datasource.DataSource) (*Repository, error) {
r := &Repository{
ds: ds,
contactCache: make(map[string]*model.Contact),
aliasToContact: make(map[string]*model.Contact),
remarkToContact: make(map[string]*model.Contact),
nickNameToContact: make(map[string]*model.Contact),
aliasToContact: make(map[string][]*model.Contact),
remarkToContact: make(map[string][]*model.Contact),
nickNameToContact: make(map[string][]*model.Contact),
chatRoomUserToInfo: make(map[string]*model.Contact),
contactList: make([]string, 0),
aliasList: make([]string, 0),
remarkList: make([]string, 0),
nickNameList: make([]string, 0),
chatRoomCache: make(map[string]*model.ChatRoom),
remarkToChatRoom: make(map[string]*model.ChatRoom),
nickNameToChatRoom: make(map[string]*model.ChatRoom),
remarkToChatRoom: make(map[string][]*model.ChatRoom),
nickNameToChatRoom: make(map[string][]*model.ChatRoom),
chatRoomList: make([]string, 0),
chatRoomRemark: make([]string, 0),
chatRoomNickName: make([]string, 0),
@@ -61,6 +64,9 @@ func New(ds datasource.DataSource) (*Repository, error) {
return nil, errors.InitCacheFailed(err)
}
ds.SetCallback("contact", r.contactCallback)
ds.SetCallback("chatroom", r.chatroomCallback)
return r, nil
}
@@ -79,6 +85,26 @@ func (r *Repository) initCache(ctx context.Context) error {
return nil
}
func (r *Repository) contactCallback(event fsnotify.Event) error {
if !event.Op.Has(fsnotify.Create) {
return nil
}
if err := r.initContactCache(context.Background()); err != nil {
log.Err(err).Msgf("Failed to reinitialize contact cache: %s", event.Name)
}
return nil
}
func (r *Repository) chatroomCallback(event fsnotify.Event) error {
if !event.Op.Has(fsnotify.Create) {
return nil
}
if err := r.initChatRoomCache(context.Background()); err != nil {
log.Err(err).Msgf("Failed to reinitialize contact cache: %s", event.Name)
}
return nil
}
// Close 实现 Repository 接口的 Close 方法
func (r *Repository) Close() error {
return r.ds.Close()

View File

@@ -57,11 +57,11 @@ func (w *DB) Initialize() error {
return nil
}
func (w *DB) GetMessages(start, end time.Time, talker string, limit, offset int) ([]*model.Message, error) {
func (w *DB) GetMessages(start, end time.Time, talker string, sender string, keyword string, limit, offset int) ([]*model.Message, error) {
ctx := context.Background()
// 使用 repository 获取消息
messages, err := w.repo.GetMessages(ctx, start, end, talker, limit, offset)
messages, err := w.repo.GetMessages(ctx, start, end, talker, sender, keyword, limit, offset)
if err != nil {
return nil, err
}

628
pkg/filecopy/filecopy.go Normal file
View File

@@ -0,0 +1,628 @@
package filecopy
import (
"encoding/json"
"fmt"
"hash/fnv"
"io"
"os"
"path/filepath"
"strings"
"sync"
"time"
)
var (
// Singleton locks to ensure only one thread processes the same file at a time
fileOperationLocks = make(map[string]*sync.Mutex)
locksMutex = sync.RWMutex{}
// Mapping from original file paths to temporary file paths
pathToTempFile = make(map[string]string)
// Metadata information for original files
fileMetadata = make(map[string]fileMetaInfo)
// Track old versions of temporary files for each original file
oldVersions = make(map[string]string)
mapMutex = sync.RWMutex{}
// Temporary directory
tempDir string
// Path to the mapping file
mappingFilePath string
// Channel for delayed file deletion
fileDeletionChan = make(chan FileDeletion, 1000)
// Default deletion delay time (30 seconds)
DefaultDeletionDelay = 30 * time.Second
)
type FileDeletion struct {
Path string
Time time.Time
}
// File metadata information
type fileMetaInfo struct {
ModTime time.Time `json:"mod_time"`
Size int64 `json:"size"`
}
// Persistent mapping information
type persistentMapping struct {
OriginalPath string `json:"original_path"`
TempPath string `json:"temp_path"`
Metadata fileMetaInfo `json:"metadata"`
}
// Initialize temporary directory
func initTempDir() {
// Get process name to create a unique temporary directory
procName := getProcessName()
tempDir = filepath.Join(os.TempDir(), "filecopy_"+procName)
if err := os.MkdirAll(tempDir, 0755); err != nil {
tempDir = filepath.Join(os.TempDir(), "filecopy")
if err := os.MkdirAll(tempDir, 0755); err != nil {
tempDir = os.TempDir()
}
}
// Set mapping file path
mappingFilePath = filepath.Join(tempDir, "file_mappings.json")
// Load existing mappings if available
loadMappings()
// Scan and clean existing temporary files
cleanupExistingTempFiles()
}
// Get process name
func getProcessName() string {
executable, err := os.Executable()
if err != nil {
return "unknown"
}
// Extract base name (without extension)
baseName := filepath.Base(executable)
ext := filepath.Ext(baseName)
if ext != "" {
baseName = baseName[:len(baseName)-len(ext)]
}
// Clean name, keep only letters, numbers, underscores and hyphens
baseName = strings.Map(func(r rune) rune {
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '-' || r == '_' {
return r
}
return '_'
}, baseName)
return baseName
}
// Load file mappings from persistent storage
func loadMappings() {
file, err := os.Open(mappingFilePath)
if err != nil {
// It's okay if the file doesn't exist yet
return
}
defer file.Close()
var mappings []persistentMapping
decoder := json.NewDecoder(file)
if err := decoder.Decode(&mappings); err != nil {
// If the file is corrupted, we'll just start fresh
return
}
// Restore mappings
mapMutex.Lock()
defer mapMutex.Unlock()
for _, mapping := range mappings {
// Verify that both the original file and temp file still exist
origStat, origErr := os.Stat(mapping.OriginalPath)
_, tempErr := os.Stat(mapping.TempPath)
if origErr == nil && tempErr == nil {
// Check if the original file has changed since the mapping was saved
if origStat.ModTime() == mapping.Metadata.ModTime && origStat.Size() == mapping.Metadata.Size {
// The mapping is still valid
pathToTempFile[mapping.OriginalPath] = mapping.TempPath
fileMetadata[mapping.OriginalPath] = mapping.Metadata
}
}
}
}
// Save file mappings to persistent storage
func saveMappings() {
mapMutex.RLock()
defer mapMutex.RUnlock()
var mappings []persistentMapping
for origPath, tempPath := range pathToTempFile {
if meta, exists := fileMetadata[origPath]; exists {
mappings = append(mappings, persistentMapping{
OriginalPath: origPath,
TempPath: tempPath,
Metadata: meta,
})
}
}
// Create the file
file, err := os.Create(mappingFilePath)
if err != nil {
return
}
defer file.Close()
// Write the mappings
encoder := json.NewEncoder(file)
encoder.SetIndent("", " ")
if err := encoder.Encode(mappings); err != nil {
// If we can't save, just continue - it's not critical
return
}
}
// Clean up existing temporary files
func cleanupExistingTempFiles() {
files, err := os.ReadDir(tempDir)
if err != nil {
return
}
// Skip the mapping file
mappingFileName := filepath.Base(mappingFilePath)
// First, collect all files that are already in our mapping
knownFiles := make(map[string]bool)
mapMutex.RLock()
for _, tempPath := range pathToTempFile {
knownFiles[tempPath] = true
}
mapMutex.RUnlock()
// Group files by prefix (baseName_hashPrefix)
fileGroups := make(map[string][]tempFileInfo)
for _, file := range files {
if file.IsDir() {
continue
}
fileName := file.Name()
// Skip the mapping file
if fileName == mappingFileName {
continue
}
filePath := filepath.Join(tempDir, fileName)
parts := strings.Split(fileName, "_")
// Skip files that don't match our naming convention
if len(parts) < 3 {
removeFileImmediately(filePath)
continue
}
// Extract base name and hash part as key
baseName := parts[0]
hashPart := parts[1]
groupKey := baseName + "_" + hashPart
// Extract timestamp
timeStr := strings.Split(parts[2], ".")[0] // Remove extension part
var timestamp int64
if _, err := fmt.Sscanf(timeStr, "%d", &timestamp); err != nil {
removeFileImmediately(filePath)
continue
}
// Add file info to corresponding group
fileGroups[groupKey] = append(fileGroups[groupKey], tempFileInfo{
path: filePath,
timestamp: timestamp,
})
}
// Process each group of files, keep only the newest one
for _, fileInfos := range fileGroups {
if len(fileInfos) == 0 {
continue
}
// Find the newest file
var newestFile tempFileInfo
for _, fileInfo := range fileInfos {
if fileInfo.timestamp > newestFile.timestamp {
newestFile = fileInfo
}
}
// Delete all files except the newest one
for _, fileInfo := range fileInfos {
if fileInfo.path != newestFile.path {
// If this file is already in our mapping, keep it
if knownFiles[fileInfo.path] {
continue
}
removeFileImmediately(fileInfo.path)
}
}
}
}
// Temporary file information
type tempFileInfo struct {
path string
timestamp int64
}
// Get file lock
func getFileLock(path string) *sync.Mutex {
locksMutex.RLock()
lock, exists := fileOperationLocks[path]
locksMutex.RUnlock()
if exists {
return lock
}
locksMutex.Lock()
defer locksMutex.Unlock()
// Check again, might have been created while we were acquiring the write lock
lock, exists = fileOperationLocks[path]
if !exists {
lock = &sync.Mutex{}
fileOperationLocks[path] = lock
}
return lock
}
// GetTempCopy returns a temporary copy path of the original file
// If the file hasn't changed since the last copy, returns the existing copy
func GetTempCopy(originalPath string) (string, error) {
// Get the operation lock for this file to ensure thread safety
fileLock := getFileLock(originalPath)
fileLock.Lock()
defer fileLock.Unlock()
// Check if original file exists
stat, err := os.Stat(originalPath)
if err != nil {
return "", fmt.Errorf("original file does not exist: %w", err)
}
// Current file info
currentInfo := fileMetaInfo{
ModTime: stat.ModTime(),
Size: stat.Size(),
}
// Check existing mapping
mapMutex.RLock()
tempPath, pathExists := pathToTempFile[originalPath]
cachedInfo, infoExists := fileMetadata[originalPath]
mapMutex.RUnlock()
// If we have an existing temp file and original file hasn't changed, return it
if pathExists && infoExists {
fileChanged := currentInfo.ModTime.After(cachedInfo.ModTime) ||
currentInfo.Size != cachedInfo.Size
if !fileChanged {
// Verify temp file still exists
if _, err := os.Stat(tempPath); err == nil {
// Try to open file to verify accessibility
if file, err := os.Open(tempPath); err == nil {
file.Close()
return tempPath, nil
}
}
}
}
// Generate new temp file path
fileName := filepath.Base(originalPath)
fileExt := filepath.Ext(fileName)
baseName := fileName[:len(fileName)-len(fileExt)]
if baseName == "" {
baseName = "file" // Use default name if empty
}
// Generate hash for original path
pathHash := hashString(originalPath)
hashPrefix := getHashPrefix(pathHash, 8)
// Format: basename_pathhash_timestamp.ext
timestamp := time.Now().UnixNano()
tempPath = filepath.Join(tempDir,
fmt.Sprintf("%s_%s_%d%s",
baseName,
hashPrefix,
timestamp,
fileExt))
// Copy file (with retry mechanism)
if err := copyFileWithRetry(originalPath, tempPath, 3); err != nil {
return "", err
}
// Update mappings
mapMutex.Lock()
oldPath := pathToTempFile[originalPath]
// If there's an old path and it's different, move it to old versions and schedule for deletion
if oldPath != "" && oldPath != tempPath {
// First clean up previous old version (if any)
if oldVersionPath, hasOldVersion := oldVersions[originalPath]; hasOldVersion && oldVersionPath != oldPath {
removeFileImmediately(oldVersionPath)
}
// Set current version as old version
oldVersions[originalPath] = oldPath
scheduleForDeletion(oldPath)
}
// Update to new temp file
pathToTempFile[originalPath] = tempPath
fileMetadata[originalPath] = currentInfo
mapMutex.Unlock()
// Save mappings to persistent storage
go saveMappings()
// Immediately clean up any other related temp files
go cleanupRelatedTempFiles(originalPath, tempPath, oldPath)
return tempPath, nil
}
// Immediately clean up other temp files related to the specified original file
func cleanupRelatedTempFiles(originalPath, currentTempPath, knownOldPath string) {
// Extract hash prefix of original file to match related files
fileName := filepath.Base(originalPath)
fileExt := filepath.Ext(fileName)
baseName := fileName[:len(fileName)-len(fileExt)]
if baseName == "" {
baseName = "file"
}
pathHash := hashString(originalPath)
hashPrefix := getHashPrefix(pathHash, 8)
// File name prefix pattern
filePrefix := baseName + "_" + hashPrefix
currentTempPathNoExt := strings.TrimSuffix(currentTempPath, filepath.Ext(currentTempPath))
knownOldPathNoExt := strings.TrimSuffix(knownOldPath, filepath.Ext(knownOldPath))
files, err := os.ReadDir(tempDir)
if err != nil {
return
}
for _, file := range files {
if file.IsDir() {
continue
}
fileName := file.Name()
// Skip the mapping file
if fileName == filepath.Base(mappingFilePath) {
continue
}
filePath := filepath.Join(tempDir, fileName)
filePathNoExt := strings.TrimSuffix(filePath, filepath.Ext(filePath))
// Skip current file and known old version
if filePathNoExt == currentTempPathNoExt || filePathNoExt == knownOldPathNoExt {
continue
}
// If file name matches our pattern, delete it immediately
if strings.HasPrefix(fileName, filePrefix) {
removeFileImmediately(filePath)
}
}
}
// Immediately delete file without waiting for delay
func removeFileImmediately(path string) {
if path == "" {
return
}
// Try to delete file
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
// Silently fail if we can't delete
}
}
// Schedule file for delayed deletion
func scheduleForDeletion(path string) {
if path == "" {
return
}
// Check if file exists
if _, err := os.Stat(path); os.IsNotExist(err) {
return
}
// Put file in deletion channel
select {
case fileDeletionChan <- FileDeletion{Path: path, Time: time.Now().Add(DefaultDeletionDelay)}:
// Successfully scheduled
default:
// If channel is full, delete file immediately
removeFileImmediately(path)
}
}
// File deletion handler
func fileDeletionHandler() {
for {
// Get file to delete from channel
file := <-fileDeletionChan
if !time.Now().After(file.Time) {
time.Sleep(time.Until(file.Time))
}
// Ensure file is not in active mappings
isActive := false
mapMutex.RLock()
for _, activePath := range pathToTempFile {
if activePath == file.Path {
isActive = true
break
}
}
mapMutex.RUnlock()
if isActive {
continue
}
// Delete file
removeFileImmediately(file.Path)
}
}
// CleanupTempFiles cleans up unused temporary files
func CleanupTempFiles() {
files, err := os.ReadDir(tempDir)
if err != nil {
return
}
// Skip the mapping file
mappingFileName := filepath.Base(mappingFilePath)
// Get current active temp file paths and old version paths
mapMutex.RLock()
activeTempFiles := make(map[string]bool)
for _, tempFilePath := range pathToTempFile {
tempFilePath = strings.TrimSuffix(tempFilePath, filepath.Ext(tempFilePath))
activeTempFiles[tempFilePath] = true
}
for _, oldVersionPath := range oldVersions {
oldVersionPath = strings.TrimSuffix(oldVersionPath, filepath.Ext(oldVersionPath))
activeTempFiles[oldVersionPath] = true
}
mapMutex.RUnlock()
// Schedule deletion of inactive temp files
for _, file := range files {
if file.IsDir() {
continue
}
fileName := file.Name()
// Skip the mapping file
if fileName == mappingFileName {
continue
}
tempFilePath := filepath.Join(tempDir, fileName)
tempFilePath = strings.TrimSuffix(tempFilePath, filepath.Ext(tempFilePath))
if !activeTempFiles[tempFilePath] {
scheduleForDeletion(tempFilePath)
}
}
}
// Copy file with retry mechanism
func copyFileWithRetry(src, dst string, maxRetries int) error {
var err error
for i := 0; i < maxRetries; i++ {
err = copyFile(src, dst)
if err == nil {
return nil
}
// Wait before retrying
time.Sleep(time.Duration(100*(i+1)) * time.Millisecond)
}
return fmt.Errorf("failed to copy file after %d attempts: %w", maxRetries, err)
}
// Copy file
func copyFile(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return fmt.Errorf("failed to open source file: %w", err)
}
defer in.Close()
out, err := os.Create(dst)
if err != nil {
return fmt.Errorf("failed to create destination file: %w", err)
}
defer func() {
cerr := out.Close()
if err == nil && cerr != nil {
err = fmt.Errorf("failed to close destination file: %w", cerr)
}
}()
// Use buffered copy for better performance
buf := make([]byte, 256*1024) // 256KB buffer
if _, err = io.CopyBuffer(out, in, buf); err != nil {
return fmt.Errorf("failed to copy file contents: %w", err)
}
return out.Sync()
}
// Generate hash for string
func hashString(s string) string {
h := fnv.New32a()
h.Write([]byte(s))
return fmt.Sprintf("%x", h.Sum32())
}
// Safely get hash prefix, avoid index out of bounds
func getHashPrefix(hash string, length int) string {
if len(hash) <= length {
return hash
}
return hash[:length]
}
// Initialize temp directory and start background cleanup
func init() {
// Initialize temp directory and scan existing files
initTempDir()
// Start multiple file deletion handlers
for i := 0; i < 2; i++ {
go fileDeletionHandler()
}
// Start periodic cleanup routine
go func() {
for {
time.Sleep(30 * time.Second)
CleanupTempFiles()
// Also periodically save mappings
saveMappings()
}
}()
}

View File

@@ -0,0 +1,182 @@
package filemonitor
import (
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"strings"
"sync"
"github.com/fsnotify/fsnotify"
"github.com/rs/zerolog/log"
)
// FileChangeCallback defines the callback function signature for file change events
type FileChangeCallback func(event fsnotify.Event) error
// FileGroup represents a group of files with the same processing logic
type FileGroup struct {
ID string // Unique identifier
RootDir string // Root directory
Pattern *regexp.Regexp // File matching pattern
PatternStr string // Original pattern string for rebuilding
Blacklist []string // Blacklist patterns
Callbacks []FileChangeCallback // File change callbacks
mutex sync.RWMutex // Concurrency control
}
// NewFileGroup creates a new file group
func NewFileGroup(id, rootDir, pattern string, blacklist []string) (*FileGroup, error) {
// Compile the regular expression
re, err := regexp.Compile(pattern)
if err != nil {
return nil, fmt.Errorf("invalid pattern '%s': %w", pattern, err)
}
// Normalize root directory path
rootDir = filepath.Clean(rootDir)
return &FileGroup{
ID: id,
RootDir: rootDir,
Pattern: re,
PatternStr: pattern,
Blacklist: blacklist,
Callbacks: []FileChangeCallback{},
}, nil
}
// AddCallback adds a callback function to the file group
func (fg *FileGroup) AddCallback(callback FileChangeCallback) {
fg.mutex.Lock()
defer fg.mutex.Unlock()
fg.Callbacks = append(fg.Callbacks, callback)
}
// RemoveCallback removes a callback function from the file group
func (fg *FileGroup) RemoveCallback(callbackToRemove FileChangeCallback) bool {
fg.mutex.Lock()
defer fg.mutex.Unlock()
for i, callback := range fg.Callbacks {
// Compare function addresses
if fmt.Sprintf("%p", callback) == fmt.Sprintf("%p", callbackToRemove) {
// Remove the callback
fg.Callbacks = append(fg.Callbacks[:i], fg.Callbacks[i+1:]...)
return true
}
}
return false
}
// Match checks if a file path matches this group's criteria
func (fg *FileGroup) Match(path string) bool {
// Normalize paths for comparison
path = filepath.Clean(path)
rootDir := filepath.Clean(fg.RootDir)
// Check if path is under root directory
// Use filepath.Rel to handle path comparison safely across different OSes
relPath, err := filepath.Rel(rootDir, path)
if err != nil || strings.HasPrefix(relPath, "..") {
return false
}
// Check if file matches pattern
if !fg.Pattern.MatchString(filepath.Base(path)) {
return false
}
// Check blacklist
for _, blackItem := range fg.Blacklist {
if strings.Contains(relPath, blackItem) {
return false
}
}
return true
}
// List returns a list of files in the group (real-time scan)
func (fg *FileGroup) List() ([]string, error) {
files := []string{}
// Scan directory for matching files using fs.WalkDir
err := fs.WalkDir(os.DirFS(fg.RootDir), ".", func(path string, d fs.DirEntry, err error) error {
if err != nil {
return fs.SkipDir
}
// Skip directories
if d.IsDir() {
return nil
}
// Convert relative path to absolute
absPath := filepath.Join(fg.RootDir, path)
// Use Match function to check if file belongs to this group
if fg.Match(absPath) {
files = append(files, absPath)
}
return nil
})
if err != nil && !errors.Is(err, fs.ErrNotExist) {
return nil, fmt.Errorf("error listing files: %w", err)
}
return files, nil
}
// ListMatchingDirectories returns directories containing matching files
func (fg *FileGroup) ListMatchingDirectories() (map[string]bool, error) {
directories := make(map[string]bool)
// Get matching files
files, err := fg.List()
if err != nil {
return nil, err
}
// Extract directories from matching files
for _, file := range files {
dir := filepath.Dir(file)
directories[dir] = true
}
return directories, nil
}
// HandleEvent processes a file event and triggers callbacks if the file matches
func (fg *FileGroup) HandleEvent(event fsnotify.Event) {
// Check if this event is relevant for this group
if !fg.Match(event.Name) {
return
}
// Get callbacks under read lock
fg.mutex.RLock()
callbacks := make([]FileChangeCallback, len(fg.Callbacks))
copy(callbacks, fg.Callbacks)
fg.mutex.RUnlock()
// Asynchronously call callbacks
for _, callback := range callbacks {
go func(cb FileChangeCallback) {
if err := cb(event); err != nil {
log.Error().
Str("file", event.Name).
Str("op", event.Op.String()).
Err(err).
Msg("Callback error")
}
}(callback)
}
}

View File

@@ -0,0 +1,430 @@
package filemonitor
import (
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"sync"
"github.com/fsnotify/fsnotify"
"github.com/rs/zerolog/log"
)
// FileMonitor manages multiple file groups
type FileMonitor struct {
groups map[string]*FileGroup // Map of file groups
watcher *fsnotify.Watcher // File system watcher
watchDirs map[string]bool // Monitored directories
blacklist []string // Global blacklist patterns
mutex sync.RWMutex // Concurrency control for groups and watchDirs
stopCh chan struct{} // Stop signal
wg sync.WaitGroup // Wait group
isRunning bool // Running state flag
stateMutex sync.RWMutex // State mutex
}
func (fm *FileMonitor) Watcher() *fsnotify.Watcher {
return fm.watcher
}
// NewFileMonitor creates a new file monitor
func NewFileMonitor() *FileMonitor {
return &FileMonitor{
groups: make(map[string]*FileGroup),
watchDirs: make(map[string]bool),
blacklist: []string{},
isRunning: false,
}
}
// SetBlacklist sets the global directory blacklist
func (fm *FileMonitor) SetBlacklist(blacklist []string) {
fm.mutex.Lock()
defer fm.mutex.Unlock()
fm.blacklist = make([]string, len(blacklist))
copy(fm.blacklist, blacklist)
}
// AddGroup adds a new file group
func (fm *FileMonitor) AddGroup(group *FileGroup) error {
if group == nil {
return errors.New("group cannot be nil")
}
// First check if monitor is running
isRunning := fm.IsRunning()
// Add group to monitor
fm.mutex.Lock()
// Check if ID already exists
if _, exists := fm.groups[group.ID]; exists {
fm.mutex.Unlock()
return fmt.Errorf("group with ID '%s' already exists", group.ID)
}
// Add to monitor
fm.groups[group.ID] = group
fm.mutex.Unlock()
// If monitor is running, set up watching
if isRunning {
if err := fm.setupWatchForGroup(group); err != nil {
// Remove group on failure
fm.mutex.Lock()
delete(fm.groups, group.ID)
fm.mutex.Unlock()
return err
}
}
return nil
}
// CreateGroup creates and adds a new file group (convenience method)
func (fm *FileMonitor) CreateGroup(id, rootDir, pattern string, blacklist []string) (*FileGroup, error) {
// Create file group
group, err := NewFileGroup(id, rootDir, pattern, blacklist)
if err != nil {
return nil, err
}
// Add to monitor
if err := fm.AddGroup(group); err != nil {
return nil, err
}
return group, nil
}
// RemoveGroup removes a file group
func (fm *FileMonitor) RemoveGroup(id string) error {
fm.mutex.Lock()
defer fm.mutex.Unlock()
// Check if group exists
_, exists := fm.groups[id]
if !exists {
return fmt.Errorf("group with ID '%s' does not exist", id)
}
// Remove group
delete(fm.groups, id)
// log.Info().Str("groupID", id).Msg("Removed file group")
return nil
}
// GetGroups returns a list of all file group IDs
func (fm *FileMonitor) GetGroups() []*FileGroup {
fm.mutex.RLock()
defer fm.mutex.RUnlock()
groups := make([]*FileGroup, 0, len(fm.groups))
for _, group := range fm.groups {
groups = append(groups, group)
}
return groups
}
// GetGroup returns the specified file group
func (fm *FileMonitor) GetGroup(id string) (*FileGroup, bool) {
fm.mutex.RLock()
defer fm.mutex.RUnlock()
group, exists := fm.groups[id]
return group, exists
}
// Start starts the file monitor
func (fm *FileMonitor) Start() error {
// Check if already running
fm.stateMutex.Lock()
if fm.isRunning {
fm.stateMutex.Unlock()
return errors.New("file monitor is already running")
}
// Create new watcher
watcher, err := fsnotify.NewWatcher()
if err != nil {
fm.stateMutex.Unlock()
return fmt.Errorf("failed to create watcher: %w", err)
}
fm.watcher = watcher
// Reset stop channel
fm.stopCh = make(chan struct{})
// Get groups to monitor (without holding the state lock)
fm.mutex.RLock()
groups := make([]*FileGroup, 0, len(fm.groups))
for _, group := range fm.groups {
groups = append(groups, group)
}
fm.mutex.RUnlock()
// Reset monitored directories
fm.mutex.Lock()
fm.watchDirs = make(map[string]bool)
fm.mutex.Unlock()
// Mark as running before setting up watches
fm.isRunning = true
fm.stateMutex.Unlock()
// Set up monitoring for all groups (without holding any locks)
for _, group := range groups {
if err := fm.setupWatchForGroup(group); err != nil {
// Clean up resources on failure
_ = fm.watcher.Close()
// Reset running state
fm.stateMutex.Lock()
fm.watcher = nil
fm.isRunning = false
fm.stateMutex.Unlock()
return fmt.Errorf("failed to setup watch for group '%s': %w", group.ID, err)
}
}
// Start watch loop
fm.wg.Add(1)
go fm.watchLoop()
// log.Info().Msg("File monitor started")
return nil
}
// Stop stops the file monitor
func (fm *FileMonitor) Stop() error {
// Check if already stopped
fm.stateMutex.Lock()
if !fm.isRunning {
fm.stateMutex.Unlock()
return errors.New("file monitor is not running")
}
// Get watcher reference before changing state
watcher := fm.watcher
// Send stop signal
close(fm.stopCh)
// Mark as not running
fm.isRunning = false
fm.stateMutex.Unlock()
// Wait for all goroutines to exit
fm.wg.Wait()
// Close watcher
if watcher != nil {
if err := watcher.Close(); err != nil {
return fmt.Errorf("failed to close watcher: %w", err)
}
fm.stateMutex.Lock()
fm.watcher = nil
fm.stateMutex.Unlock()
}
// log.Info().Msg("File monitor stopped")
return nil
}
// IsRunning returns whether the file monitor is running
func (fm *FileMonitor) IsRunning() bool {
fm.stateMutex.RLock()
defer fm.stateMutex.RUnlock()
return fm.isRunning
}
// addWatchDir adds a directory to monitoring
func (fm *FileMonitor) addWatchDir(dirPath string) error {
// Check global blacklist first
fm.mutex.RLock()
for _, pattern := range fm.blacklist {
if strings.Contains(dirPath, pattern) {
fm.mutex.RUnlock()
log.Debug().Str("dir", dirPath).Msg("Skipping blacklisted directory")
return nil
}
}
fm.mutex.RUnlock()
fm.mutex.Lock()
defer fm.mutex.Unlock()
// Check if directory is already being monitored
if _, watched := fm.watchDirs[dirPath]; watched {
return nil // Already monitored, no need to add again
}
// Add to monitoring
if err := fm.watcher.Add(dirPath); err != nil {
return fmt.Errorf("failed to watch directory '%s': %w", dirPath, err)
}
fm.watchDirs[dirPath] = true
// log.Debug().Str("dir", dirPath).Msg("Added watch for directory")
return nil
}
// setupWatchForGroup sets up monitoring for a file group
func (fm *FileMonitor) setupWatchForGroup(group *FileGroup) error {
// Check if file monitor is running
if !fm.IsRunning() {
return errors.New("file monitor is not running")
}
// Find directories containing matching files
matchingDirs, err := group.ListMatchingDirectories()
if err != nil {
return fmt.Errorf("failed to list matching directories: %w", err)
}
// Always watch the root directory to catch new files
rootDir := filepath.Clean(group.RootDir)
if err := fm.addWatchDir(rootDir); err != nil {
return err
}
// Watch directories containing matching files
for dir := range matchingDirs {
if err := fm.addWatchDir(dir); err != nil {
return err
}
}
return nil
}
// RefreshWatches updates the watched directories based on current matching files
func (fm *FileMonitor) RefreshWatches() error {
// Check if file monitor is running
if !fm.IsRunning() {
return errors.New("file monitor is not running")
}
// Get groups to refresh
fm.mutex.RLock()
groups := make([]*FileGroup, 0, len(fm.groups))
for _, group := range fm.groups {
groups = append(groups, group)
}
fm.mutex.RUnlock()
// Reset watched directories
fm.mutex.Lock()
oldWatchDirs := fm.watchDirs
fm.watchDirs = make(map[string]bool)
fm.mutex.Unlock()
// Setup watches for each group
for _, group := range groups {
if err := fm.setupWatchForGroup(group); err != nil {
return fmt.Errorf("failed to refresh watches for group '%s': %w", group.ID, err)
}
}
// Remove watches for directories no longer needed
for dir := range oldWatchDirs {
fm.mutex.RLock()
_, stillWatched := fm.watchDirs[dir]
fm.mutex.RUnlock()
if !stillWatched && fm.watcher != nil {
_ = fm.watcher.Remove(dir)
log.Debug().Str("dir", dir).Msg("Removed watch for directory")
}
}
return nil
}
// watchLoop monitors for file system events
func (fm *FileMonitor) watchLoop() {
defer fm.wg.Done()
for {
select {
case <-fm.stopCh:
return
case event, ok := <-fm.watcher.Events:
if !ok {
// Channel closed, exit loop
return
}
// Handle directory creation events to add new watches
info, err := os.Stat(event.Name)
if err == nil && info.IsDir() && event.Op&(fsnotify.Create|fsnotify.Rename) != 0 {
// Add new directory to monitoring
if err := fm.addWatchDir(event.Name); err != nil {
log.Error().
Str("dir", event.Name).
Err(err).
Msg("Error watching new directory")
}
continue
}
// For file creation/modification, check if we need to watch its directory
if event.Op&(fsnotify.Create|fsnotify.Write) != 0 {
// Check if this file matches any group
shouldWatch := false
fm.mutex.RLock()
for _, group := range fm.groups {
if group.Match(event.Name) {
shouldWatch = true
break
}
}
fm.mutex.RUnlock()
// If file matches, ensure its directory is watched
if shouldWatch {
dir := filepath.Dir(event.Name)
if err := fm.addWatchDir(dir); err != nil {
log.Error().
Str("dir", dir).
Err(err).
Msg("Error watching directory of matching file")
}
}
}
// Forward event to all groups
fm.forwardEventToGroups(event)
case err, ok := <-fm.watcher.Errors:
if !ok {
// Channel closed, exit loop
return
}
log.Error().Err(err).Msg("Watcher error")
}
}
}
// forwardEventToGroups forwards file events to matching groups
func (fm *FileMonitor) forwardEventToGroups(event fsnotify.Event) {
// Get a copy of groups to avoid holding lock during processing
fm.mutex.RLock()
groupsCopy := make([]*FileGroup, 0, len(fm.groups))
for _, group := range fm.groups {
groupsCopy = append(groupsCopy, group)
}
fm.mutex.RUnlock()
// Forward to all groups - each group will check if the event is relevant
for _, group := range groupsCopy {
group.HandleEvent(event)
}
}

36
pkg/util/silk/silk.go Normal file
View File

@@ -0,0 +1,36 @@
package silk
import (
"fmt"
"github.com/sjzar/go-lame"
"github.com/sjzar/go-silk"
)
func Silk2MP3(data []byte) ([]byte, error) {
sd := silk.SilkInit()
defer sd.Close()
pcmdata := sd.Decode(data)
if len(pcmdata) == 0 {
return nil, fmt.Errorf("silk decode failed")
}
le := lame.Init()
defer le.Close()
le.SetInSamplerate(24000)
le.SetOutSamplerate(24000)
le.SetNumChannels(1)
le.SetBitrate(16)
// IMPORTANT!
le.InitParams()
mp3data := le.Encode(pcmdata)
if len(mp3data) == 0 {
return nil, fmt.Errorf("mp3 encode failed")
}
return mp3data, nil
}

View File

@@ -3,6 +3,7 @@ package util
import (
"fmt"
"strconv"
"strings"
"unicode"
"unicode/utf8"
)
@@ -45,3 +46,26 @@ func IsNumeric(s string) bool {
func SplitInt64ToTwoInt32(input int64) (int64, int64) {
return input & 0xFFFFFFFF, input >> 32
}
func Str2List(str string, sep string) []string {
list := make([]string, 0)
if str == "" {
return list
}
listMap := make(map[string]bool)
for _, elem := range strings.Split(str, sep) {
elem = strings.TrimSpace(elem)
if len(elem) == 0 {
continue
}
if _, ok := listMap[elem]; ok {
continue
}
listMap[elem] = true
list = append(list, elem)
}
return list
}

View File

@@ -582,8 +582,8 @@ func adjustStartTime(t time.Time, g TimeGranularity) time.Time {
func adjustEndTime(t time.Time, g TimeGranularity) time.Time {
switch g {
case GranularitySecond, GranularityMinute, GranularityHour:
// 对于精确到秒/分钟/小时的时间,设置为当天结束
return time.Date(t.Year(), t.Month(), t.Day(), 23, 59, 59, 999999999, t.Location())
// 对于精确到秒/分钟/小时的时间,保持原样
return t
case GranularityDay:
// 精确到天,设置为当天结束
return time.Date(t.Year(), t.Month(), t.Day(), 23, 59, 59, 999999999, t.Location())
@@ -634,3 +634,25 @@ func isValidDate(year, month, day int) bool {
return day <= daysInMonth
}
func PerfectTimeFormat(start time.Time, end time.Time) string {
endTime := end
// 如果结束时间是某一天的 0 点整,将其减去 1 秒,视为前一天的结束
if endTime.Hour() == 0 && endTime.Minute() == 0 && endTime.Second() == 0 && endTime.Nanosecond() == 0 {
endTime = endTime.Add(-time.Second) // 减去 1 秒
}
// 判断是否跨年
if start.Year() != endTime.Year() {
return "2006-01-02 15:04:05" // 完整格式,包含年月日时分秒
}
// 判断是否跨天(但在同一年内)
if start.YearDay() != endTime.YearDay() {
return "01-02 15:04:05" // 月日时分秒格式
}
// 在同一天内
return "15:04:05" // 只显示时分秒
}