diff --git a/README.md b/README.md index f1b3dbb..2634d2a 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ → 把链接发给任意 AI → AI 读取目录并按描述访问具体文档 ``` -> 状态:**开发中(Phase 1 — 项目初始化)**。完整架构与技术决策见 [docs/technical-review.md](docs/technical-review.md)。 +> 状态:**MVP 开发完成(Phase 1-14)**。完整架构与技术决策见 [docs/technical-review.md](docs/technical-review.md);需求清单见 [docs/requirements.md](docs/requirements.md)。 ## 技术栈(MVP v1) @@ -17,7 +17,7 @@ | 前端 | Vue 3 · TypeScript · Vite · Element Plus · Pinia | | 存储 | SQLite(元数据)+ 本地文件系统(原始文件 + Markdown) | | 解析 | MarkItDown(优先)· PyMuPDF(PDF fallback)· python-docx(DOCX fallback) | -| 搜索 | SQLite FTS5 | +| 搜索 | SQLite LIKE(可升级 FTS5) | ## 目录结构 @@ -62,25 +62,211 @@ npm run dev - 访问:http://localhost:5173 - Vite 代理:`/api` 和 `/k` → `localhost:8000` -## 生产部署(Ubuntu + Docker) +### 测试 ```bash +cd backend +python -m pytest tests/ -v +``` + +## 生产部署(Ubuntu + Docker) + +### 1. 安装 Docker + +```bash +# 更新系统 +sudo apt update && sudo apt upgrade -y + +# 安装 Docker +curl -fsSL https://get.docker.com | sh +sudo usermod -aG docker $USER +newgrp docker + +# 验证 +docker --version +docker compose version +``` + +### 2. 克隆项目 + +```bash +sudo mkdir -p /opt/ai-knowledge-link +sudo chown $USER:$USER /opt/ai-knowledge-link git clone /opt/ai-knowledge-link cd /opt/ai-knowledge-link -cp .env.example .env # 修改全部密钥 +``` + +### 3. 配置环境 + +```bash +cp .env.example .env + +# 编辑 .env,修改以下关键配置: +nano .env +``` + +必须修改的配置: + +```env +# 生成密钥:python -c "import secrets; print(secrets.token_urlsafe(48))" +SECRET_KEY= +ENVIRONMENT=production +``` + +### 4. 创建数据目录 + +```bash mkdir -p data +``` + +### 5. 启动服务 + +```bash docker compose up -d --build ``` -- 数据持久化:`./data/` 宿主机 volume,删除容器不丢失 -- 备份:SQLite `.backup` + tar 用户文件目录 +### 6. 验证部署 + +```bash +# 检查容器状态 +docker compose ps + +# 检查日志 +docker compose logs -f + +# 健康检查 +curl http://localhost/api/healthz +``` + +### 7. 访问 + +- 前端:http://your-server-ip +- API 文档:http://your-server-ip/api/docs(仅开发环境) +- 公共页面:http://your-server-ip/k/{token} + +### 8. 更新部署 + +```bash +cd /opt/ai-knowledge-link +git pull +docker compose up -d --build +``` + +数据不会因为更新丢失(`./data/` 是宿主机 volume)。 + +### 9. 备份 + +```bash +# 备份 SQLite 数据库 +sqlite3 data/app.db ".backup data/backup/app_$(date +%Y%m%d).db" + +# 备份用户文件 +tar czf data/backup/files_$(date +%Y%m%d).tar.gz data/users/ + +# 保留最近 7 天备份 +find data/backup/ -name "*.db" -mtime +7 -delete +find data/backup/ -name "*.tar.gz" -mtime +7 -delete +``` + +### 10. 恢复 + +```bash +# 恢复数据库 +cp data/backup/app_20260901.db data/app.db + +# 恢复用户文件 +tar xzf data/backup/files_20260901.tar.gz -C / +``` + +## Nginx 配置(生产) + +如果使用独立 Nginx(非 Docker),参考 `nginx/nginx.conf`: + +```nginx +server { + listen 80; + server_name your-domain.com; + + client_max_body_size 25M; + + # 前端静态文件 + location / { + root /opt/ai-knowledge-link/frontend/dist; + try_files $uri $uri/ /index.html; + } + + # API 代理 + location /api/ { + proxy_pass http://127.0.0.1:8000; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_read_timeout 60s; + } + + # 公共 AI 页面代理 + location /k/ { + proxy_pass http://127.0.0.1:8000; + proxy_set_header Host $host; + proxy_set_header X-Real-IP $remote_addr; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + add_header Cache-Control "private, no-cache, no-store" always; + add_header X-Robots-Tag "noindex, nofollow, noarchive" always; + } +} +``` ## 安全须知 - `.env` 携带全部密钥,**绝不提交 Git** - 知识库 Secret URL 即访问凭证(链接即钥匙),请勿公开传播 - 公共 AI 页面不依赖 JS/Cookie/登录;含 `noindex` meta + robots.txt 屏蔽 +- 生产环境必须设置 `ENVIRONMENT=production`(启用 Cookie Secure、禁用 Swagger) + +## API 概览 + +### 管理端(需登录) + +| 方法 | 路径 | 说明 | +|---|---|---| +| POST | /api/auth/register | 注册 | +| POST | /api/auth/login | 登录 | +| POST | /api/auth/logout | 登出 | +| GET | /api/auth/me | 当前用户信息 | +| PATCH | /api/auth/me | 修改密码 | +| GET | /api/auth/storage | 存储用量 | +| GET | /api/knowledge-bases | 知识库列表 | +| POST | /api/knowledge-bases | 创建知识库 | +| GET | /api/knowledge-bases/{id} | 知识库详情 | +| PUT | /api/knowledge-bases/{id} | 编辑知识库 | +| DELETE | /api/knowledge-bases/{id} | 删除知识库 | +| POST | /api/knowledge-bases/{id}/regenerate-token | 重新生成链接 | +| POST | /api/knowledge-bases/{id}/enable | 启用链接 | +| POST | /api/knowledge-bases/{id}/disable | 禁用链接 | +| GET | /api/knowledge-bases/{id}/link | 获取完整链接 | +| GET | /api/knowledge-bases/{id}/categories | 分类列表 | +| POST | /api/knowledge-bases/{id}/categories | 创建分类 | +| GET | /api/documents?kb_id= | 文档列表 | +| POST | /api/documents/upload | 上传文档 | +| GET | /api/documents/{id} | 文档详情 | +| PUT | /api/documents/{id} | 编辑文档 | +| DELETE | /api/documents/{id} | 删除文档 | +| POST | /api/documents/{id}/reprocess | 重新解析 | + +### 公共 AI 端(无需登录) + +| 方法 | 路径 | 说明 | +|---|---|---| +| GET | /k/{token} | 知识库首页(HTML) | +| GET | /k/{token}.md | 知识库首页(Markdown) | +| GET | /k/{token}.txt | 知识库首页(纯文本) | +| GET | /k/{token}.json | 知识库首页(JSON) | +| GET | /k/{token}/search?q= | 搜索文档 | +| GET | /k/{token}/doc/{doc_token} | 文档页面(HTML) | +| GET | /k/{token}/doc/{doc_token}.md | 文档(Markdown) | +| GET | /k/{token}/doc/{doc_token}.txt | 文档(纯文本) | ## 文档 -- [技术审查报告](docs/technical-review.md) — 架构、ER、API、安全模型、风险清单、6 个扩展接口 \ No newline at end of file +- [技术审查报告](docs/technical-review.md) — 架构、ER、API、安全模型、风险清单、6 个扩展接口 +- [需求清单](docs/requirements.md) — 完整需求与完成状态 diff --git a/backend/app/api/categories.py b/backend/app/api/categories.py new file mode 100644 index 0000000..200cc79 --- /dev/null +++ b/backend/app/api/categories.py @@ -0,0 +1,104 @@ +"""文档分类 API 路由。""" + +from fastapi import APIRouter, Depends +from pydantic import BaseModel, Field +from sqlalchemy.orm import Session + +from app.api.deps import get_current_user, get_db +from app.core.errors import NotFoundError +from app.models.document_category import DocumentCategory +from app.models.user import User +from app.repositories.kb_repo import KnowledgeBaseRepository + +router = APIRouter(prefix="/knowledge-bases/{kb_id}/categories", tags=["categories"]) + + +class CategoryRequest(BaseModel): + name: str = Field(min_length=1, max_length=255) + sort_order: int = 0 + + +class CategoryResponse(BaseModel): + id: str + name: str + sort_order: int + + model_config = {"from_attributes": True} + + +@router.get("", response_model=list[CategoryResponse]) +def list_categories( + kb_id: str, + user: User = Depends(get_current_user), + db: Session = Depends(get_db), +) -> list[CategoryResponse]: + _check_kb_owner(kb_id, user, db) + from sqlalchemy import select + + stmt = ( + select(DocumentCategory) + .where(DocumentCategory.knowledge_base_id == kb_id) + .order_by(DocumentCategory.sort_order) + ) + cats = list(db.scalars(stmt).all()) + return [CategoryResponse.model_validate(c) for c in cats] + + +@router.post("", response_model=CategoryResponse, status_code=201) +def create_category( + kb_id: str, + body: CategoryRequest, + user: User = Depends(get_current_user), + db: Session = Depends(get_db), +) -> CategoryResponse: + _check_kb_owner(kb_id, user, db) + cat = DocumentCategory( + knowledge_base_id=kb_id, + name=body.name, + sort_order=body.sort_order, + ) + db.add(cat) + db.commit() + db.refresh(cat) + return CategoryResponse.model_validate(cat) + + +@router.put("/{cat_id}", response_model=CategoryResponse) +def update_category( + kb_id: str, + cat_id: str, + body: CategoryRequest, + user: User = Depends(get_current_user), + db: Session = Depends(get_db), +) -> CategoryResponse: + _check_kb_owner(kb_id, user, db) + cat = db.get(DocumentCategory, cat_id) + if cat is None or cat.knowledge_base_id != kb_id: + raise NotFoundError("分类不存在。") + cat.name = body.name + cat.sort_order = body.sort_order + db.commit() + db.refresh(cat) + return CategoryResponse.model_validate(cat) + + +@router.delete("/{cat_id}", status_code=204) +def delete_category( + kb_id: str, + cat_id: str, + user: User = Depends(get_current_user), + db: Session = Depends(get_db), +) -> None: + _check_kb_owner(kb_id, user, db) + cat = db.get(DocumentCategory, cat_id) + if cat is None or cat.knowledge_base_id != kb_id: + raise NotFoundError("分类不存在。") + db.delete(cat) + db.commit() + + +def _check_kb_owner(kb_id: str, user: User, db: Session) -> None: + repo = KnowledgeBaseRepository(db) + kb = repo.get_by_id(kb_id) + if kb is None or kb.user_id != user.id or kb.status == "DELETED": + raise NotFoundError("知识库不存在。") diff --git a/backend/app/main.py b/backend/app/main.py index d936741..6d34ef2 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -11,6 +11,7 @@ from fastapi import FastAPI from fastapi.middleware.cors import CORSMiddleware from app.api.auth import router as auth_router +from app.api.categories import router as cat_router from app.api.documents import router as doc_router from app.api.health import router as health_router from app.api.knowledge_bases import router as kb_router @@ -84,6 +85,7 @@ def create_app() -> FastAPI: app.include_router(auth_router, prefix="/api", tags=["auth"]) app.include_router(kb_router, prefix="/api", tags=["knowledge-bases"]) app.include_router(doc_router, prefix="/api", tags=["documents"]) + app.include_router(cat_router, prefix="/api", tags=["categories"]) app.include_router(public_router, tags=["public"]) return app diff --git a/backend/app/public/routes.py b/backend/app/public/routes.py index 28e81d5..851d46f 100644 --- a/backend/app/public/routes.py +++ b/backend/app/public/routes.py @@ -18,11 +18,27 @@ from app.api.deps import get_db from app.core.errors import NotFoundError, RateLimitedError from app.core.rate_limit import check_rate_limit from app.core.security import decrypt_token +from app.services.access_log_service import AccessLogService from app.services.kb_public_service import KbPublicService router = APIRouter(prefix="/k", tags=["public"]) +def _log_access(db: Session, kb_id: str, path: str, request: Request, doc_id: str | None = None, req_type: str | None = None) -> None: + """记录访问日志(best-effort,不阻塞响应)。""" + try: + ua = request.headers.get("user-agent", "") + AccessLogService(db).record( + knowledge_base_id=kb_id, + document_id=doc_id, + path=path, + user_agent=ua, + request_type=req_type, + ) + except Exception: + pass # 日志失败不影响响应 + + def _rate_limit(request: Request, token: str) -> None: """限流检查。""" ip = request.client.host if request.client else "unknown" @@ -51,6 +67,7 @@ def kb_index_markdown( _rate_limit(request, token) svc = KbPublicService(db) kb = svc.get_kb_by_token(token) + _log_access(db, kb.id, f"/k/{token}.md", request, req_type="md") docs, _ = svc.list_documents(kb) lines = [f"# {kb.name}", ""] @@ -87,6 +104,7 @@ def kb_index_text( _rate_limit(request, token) svc = KbPublicService(db) kb = svc.get_kb_by_token(token) + _log_access(db, kb.id, f"/k/{token}.txt", request, req_type="txt") docs, _ = svc.list_documents(kb) lines = [kb.name, "=" * len(kb.name), ""] @@ -119,6 +137,7 @@ def kb_index_json( _rate_limit(request, token) svc = KbPublicService(db) kb = svc.get_kb_by_token(token) + _log_access(db, kb.id, f"/k/{token}.json", request, req_type="json") docs, _ = svc.list_documents(kb) doc_list = [] @@ -156,6 +175,7 @@ def kb_index_html( _rate_limit(request, token) svc = KbPublicService(db) kb = svc.get_kb_by_token(token) + _log_access(db, kb.id, f"/k/{token}", request, req_type="html") docs, total = svc.list_documents(kb, page=page, page_size=50) doc_rows = "" @@ -232,6 +252,7 @@ def doc_page_html( svc = KbPublicService(db) kb = svc.get_kb_by_token(token) doc = svc.get_document_by_token(kb, doc_token) + _log_access(db, kb.id, f"/k/{token}/doc/{doc_token}", request, doc_id=doc.id, req_type="doc_html") markdown_content = svc.get_document_markdown(doc) # Markdown → HTML(简单转换) @@ -291,6 +312,7 @@ def doc_page_markdown( svc = KbPublicService(db) kb = svc.get_kb_by_token(token) doc = svc.get_document_by_token(kb, doc_token) + _log_access(db, kb.id, f"/k/{token}/doc/{doc_token}.md", request, doc_id=doc.id, req_type="doc_md") markdown_content = svc.get_document_markdown(doc) return PlainTextResponse(content=markdown_content, media_type="text/markdown") @@ -307,6 +329,7 @@ def doc_page_text( svc = KbPublicService(db) kb = svc.get_kb_by_token(token) doc = svc.get_document_by_token(kb, doc_token) + _log_access(db, kb.id, f"/k/{token}/doc/{doc_token}.txt", request, doc_id=doc.id, req_type="doc_txt") markdown_content = svc.get_document_markdown(doc) # 去掉 Markdown 标记 @@ -332,6 +355,7 @@ def search_html( _rate_limit(request, token) svc = KbPublicService(db) kb = svc.get_kb_by_token(token) + _log_access(db, kb.id, f"/k/{token}/search?q={q}", request, req_type="search") results, total = svc.search_documents(kb, q, page=page) result_items = "" diff --git a/backend/app/services/access_log_service.py b/backend/app/services/access_log_service.py new file mode 100644 index 0000000..70e2f7f --- /dev/null +++ b/backend/app/services/access_log_service.py @@ -0,0 +1,32 @@ +"""访问日志服务。""" + +from datetime import datetime, timezone + +from app.models.access_log import AccessLog +from sqlalchemy.orm import Session + + +class AccessLogService: + def __init__(self, session: Session) -> None: + self._session = session + + def record( + self, + *, + knowledge_base_id: str, + document_id: str | None = None, + path: str, + user_agent: str | None = None, + request_type: str | None = None, + ) -> None: + """记录一次访问。""" + log = AccessLog( + knowledge_base_id=knowledge_base_id, + document_id=document_id, + path=path, + accessed_at=datetime.now(timezone.utc).isoformat(), + user_agent=user_agent[:500] if user_agent else None, + request_type=request_type, + ) + self._session.add(log) + self._session.commit() diff --git a/backend/tests/test_access_logs.py b/backend/tests/test_access_logs.py new file mode 100644 index 0000000..0b15ffe --- /dev/null +++ b/backend/tests/test_access_logs.py @@ -0,0 +1,61 @@ +"""Phase 15 访问日志测试。""" + +from fastapi.testclient import TestClient + +from app.main import app + + +def _client() -> TestClient: + return TestClient(app, raise_server_exceptions=False) + + +def test_access_log_recorded() -> None: + """访问公共页面后应有访问日志。""" + with _client() as client: + # 注册+创建知识库 + client.post("/api/auth/register", json={"username": "log_user", "email": "log@example.com", "password": "password123"}) + resp = client.post("/api/knowledge-bases", json={"name": "Log Test"}) + kb_id = resp.json()["id"] + + # 获取 AI 链接 + resp = client.get(f"/api/knowledge-bases/{kb_id}/link") + token = resp.json()["token"] + + # 访问公共页面 + client.get(f"/k/{token}") + client.get(f"/k/{token}.json") + + # 检查访问日志 + from app.core.db import get_session_factory + from sqlalchemy import text + + factory = get_session_factory() + with factory() as session: + result = session.execute(text("SELECT COUNT(*) FROM access_logs WHERE knowledge_base_id = :kb_id"), {"kb_id": kb_id}) + count = result.scalar() + assert count >= 2 + + +def test_access_log_with_kb() -> None: + """访问知识库首页应记录访问日志。""" + with _client() as client: + client.post("/api/auth/register", json={"username": "log_doc", "email": "log_doc@example.com", "password": "password123"}) + resp = client.post("/api/knowledge-bases", json={"name": "Log Test"}) + kb_id = resp.json()["id"] + + # 获取 AI 链接 + resp = client.get(f"/api/knowledge-bases/{kb_id}/link") + token = resp.json()["token"] + + # 访问公共页面 + client.get(f"/k/{token}") + + # 检查访问日志 + from app.core.db import get_session_factory + from sqlalchemy import text + + factory = get_session_factory() + with factory() as session: + result = session.execute(text("SELECT COUNT(*) FROM access_logs WHERE knowledge_base_id = :kb_id"), {"kb_id": kb_id}) + count = result.scalar() + assert count >= 1 diff --git a/backend/tests/test_categories.py b/backend/tests/test_categories.py new file mode 100644 index 0000000..d3ed78a --- /dev/null +++ b/backend/tests/test_categories.py @@ -0,0 +1,77 @@ +"""Phase 15 分类测试。""" + +from fastapi.testclient import TestClient + +from app.main import app + + +def _client() -> TestClient: + return TestClient(app, raise_server_exceptions=False) + + +def _setup(client: TestClient) -> str: + """注册+创建知识库,返回 kb_id。""" + client.post("/api/auth/register", json={ + "username": "cat_user", + "email": "cat@example.com", + "password": "password123", + }) + resp = client.post("/api/knowledge-bases", json={"name": "Cat Test KB"}) + return resp.json()["id"] + + +def test_create_category() -> None: + with _client() as client: + kb_id = _setup(client) + resp = client.post(f"/api/knowledge-bases/{kb_id}/categories", json={ + "name": "技术文档", + "sort_order": 1, + }) + assert resp.status_code == 201 + assert resp.json()["name"] == "技术文档" + assert resp.json()["sort_order"] == 1 + + +def test_list_categories() -> None: + with _client() as client: + kb_id = _setup(client) + client.post(f"/api/knowledge-bases/{kb_id}/categories", json={"name": "A", "sort_order": 2}) + client.post(f"/api/knowledge-bases/{kb_id}/categories", json={"name": "B", "sort_order": 1}) + resp = client.get(f"/api/knowledge-bases/{kb_id}/categories") + assert resp.status_code == 200 + cats = resp.json() + assert len(cats) == 2 + # 按 sort_order 排序 + assert cats[0]["name"] == "B" + assert cats[1]["name"] == "A" + + +def test_update_category() -> None: + with _client() as client: + kb_id = _setup(client) + resp = client.post(f"/api/knowledge-bases/{kb_id}/categories", json={"name": "Old"}) + cat_id = resp.json()["id"] + resp = client.put(f"/api/knowledge-bases/{kb_id}/categories/{cat_id}", json={"name": "New"}) + assert resp.status_code == 200 + assert resp.json()["name"] == "New" + + +def test_delete_category() -> None: + with _client() as client: + kb_id = _setup(client) + resp = client.post(f"/api/knowledge-bases/{kb_id}/categories", json={"name": "To Delete"}) + cat_id = resp.json()["id"] + resp = client.delete(f"/api/knowledge-bases/{kb_id}/categories/{cat_id}") + assert resp.status_code == 204 + + +def test_category_idor() -> None: + """用户A 不能操作用户B 的分类。""" + with _client() as client: + kb_id = _setup(client) + resp = client.post(f"/api/knowledge-bases/{kb_id}/categories", json={"name": "A's Cat"}) + cat_id = resp.json()["id"] + # 用户 B 登录 + client.post("/api/auth/register", json={"username": "cat_b", "email": "cat_b@example.com", "password": "password123"}) + resp = client.get(f"/api/knowledge-bases/{kb_id}/categories") + assert resp.status_code == 404 diff --git a/docs/requirements.md b/docs/requirements.md index 47af7e1..29a8bcb 100644 --- a/docs/requirements.md +++ b/docs/requirements.md @@ -24,9 +24,9 @@ | 12 | 搜索 | ✅ 完成(LIKE 搜索,可升级 FTS5) | | 13 | 安全 | 🔄 已实现:IDOR 防护、限流、XSS 防护、路径穿越防护、Session 安全 | | 14 | 前端完善 | ✅ 完成 | -| 15 | 测试 | 🔄 后端 51/51 通过;前端测试待 Phase 14 | +| 15 | 测试 | ✅ 后端 58/58 通过 | | 16 | Docker | 🔄 compose/Dockerfile 已写(Phase 1),待 Docker 环境验证 | -| 17 | Ubuntu 部署文档 | ⬜ | +| 17 | Ubuntu 部署文档 | ✅ 完成(README.md) | ---