28 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
沃巴蒂	ff22ed3a8b	feat(deepdata): 引入修改密码、文件预览组件与清洗算子增强，deepflow 改名为 deep-flow ... 本次同步 09-12 至 09-28 的代码变更，主要内容： 新增功能 - 修改密码：后端新增 ChangePasswordRequest DTO 与UserController/UserService 对应接口，前端新增 ChangePasswordDialog 并在 Header 接入入口 - 文件预览组件重构：file-preview 目录拆分出 VideoPreview（支持 blobUrl 直传）、 ImagePreview、AudioPreview，index.tsx 统一调度各类型预览器 - 新增 my_ops 算子模块：markitdown_parser、mineru_pdf_parser、text_quality_evaluator - 新增 tools/minio-file-server MinIO 文件服务 功能增强 - 数据清洗：cleaning_task_service/scheduler/routes 重构， data_juicer_executor 与 wrapper 增强，OperatorTable/LogsTable/TaskDetail 同步调整 - 数据标注：annotation sync.py、project.py 接口与schema 更新 - 数据采集、算子市场、首页与菜单路由相关页面调整 - DataJuicer 算子接入（useOperatorOperations、dj_ops_schema） 结构调整 - deepflow/ 更名为 deep-flow/（419 个文件重命名） - 新增 helm 部署编排、GitHub Actions 之外的自建镜像构建脚本 未纳入本次提交 - scripts-backup/、runtime/python-executor-backup/、components_backup/（备份目录） - *.backup*、__pycache__ 产物 - zh_core_web_sm-3.8.0-py3-none-any.whl（46MB 构建产物） - DataX/（内嵌独立仓库，远程亦未收录其源码）	2026-10-03 22:44:56 +08:00
figmar	cbb1048e18	docs: 排障表补充 callback.html 403 权限坑（600 烤进镜像）	2026-09-01 19:45:41 +08:00
figmar	20a1c8b569	docs: Casdoor SSO 部署运维手册 + 泛化 E2E 检查脚本（沉淀 31.23 踩坑）	2026-09-01 19:31:08 +08:00
figmar	f8c605b858	fix(gateway): 空 email 按用户名合成 casdoor.local 占位（email 列 NOT NULL UNIQUE）	2026-09-01 19:24:21 +08:00
figmar	13262aa4b1	fix(gateway): casdoor provisioning — 空 email 落 NULL 避免唯一键冲突；race 分支查重为空时显式报错	2026-09-01 19:22:00 +08:00
figmar	4e2ec55167	M5: 方法更新移植 —— 移除 accountOnly/定时同步，本地登录与 Casdoor SSO 并存，登录时自动建号 ... - 网关: CasdoorProperties 移除 accountOnly/sync* 配置块; SsoConfigController 移除 /sync 管理端点; UserController 移除本地登录/注册 403 - 删除 CasdoorAccountSyncService/CasdoorAdminClient/CasdoorUserDto (定时全量同步方案废弃, 由 CasdoorUserService.upsertFromCasdoor 登录时自动建号替代) - 保留既有修复: JJWT 0.12 适配(IdTokenVerifier/validateToken), updateById 修主键, RoleResolver owner/name 组名处理, WebFlux 兼容 - 前端: LoginDialog 移除 accountOnly 条件(本地登录+SSO 按钮恒显), Header 恢复注册入口恒显	2026-09-01 18:04:28 +08:00
figmar	0982dab458	M3: compose 网关服务注入 accountOnly/同步配置	2026-08-31 14:17:46 +08:00
figmar	8e5121ed0d	M3 fix: /sync 端点改 Mono 返回(WebFlux 禁 block)	2026-08-31 14:11:43 +08:00
figmar	ab1255a1eb	M3: Casdoor 账号账权全量接管 ... - accountOnly 时本地登录/注册 403,前端隐藏入口 - validateToken 增加用户存在+启用校验,禁用账号存量 JWT 立即失效 - 新增 CasdoorAdminClient(password grant 管理令牌)+ 全量账号同步服务(5 分钟定时+手动端点) - /sso/casdoor/config 暴露 accountOnly; POST /sso/casdoor/sync 手动同步(admin)	2026-08-31 13:57:58 +08:00
apescale	76e537cf34	fix(sso): updateById for existing users (IRepository.save is insert-only)	2026-08-31 12:08:13 +08:00
apescale	09b04a77fb	fix(sso): decode JWT header manually to extract kid ... jjwt 0.12 的无密钥 parse() 对已签名 JWT 抛 UnsupportedJwtException， 改为直接 base64url 解码 header 段取 kid。	2026-08-31 11:51:12 +08:00
apescale	9b2ce37ceb	feat(sso): strip owner prefix in roleKey; expose /sso/casdoor/config; casdoor env passthrough ... - RoleResolver.roleKey 取组名最后一段再匹配前缀，兼容 Casdoor user.groups 的 owner/name 全 ID - 前端 nginx 增加精确 location /sso/casdoor/config 转发网关 - docker-compose 网关服务透传 DEEPDATA_CASDOOR_* 环境变量（Spring relaxed binding）	2026-08-31 11:37:23 +08:00
apescale	c9df36be90	fix(deploy): pass DB_PASSWORD to gateway container ... Gateway datasource connects to deepdata-database with ${DB_PASSWORD}; the env var was not injected, so startup sys-param init failed with 'password authentication failed for user postgres'.	2026-08-31 07:06:26 +08:00
apescale	54e067ed54	fix(images): normalize shell scripts to LF line endings ... database image crashed at start: CRLF shebang in start.sh made the kernel look for interpreter '/bin/bash\r'. Other image Dockerfiles mask this with dos2unix at build time, but scripts/images/database/Dockerfile does not. Convert all tracked .sh files to LF and pin eol=lf via .gitattributes.	2026-08-31 06:43:31 +08:00
apescale	98b52fdb31	fix(deploy): proxy /deepflow-backend/* to backend /api/* in Docker + expose deepflow ports ... - next.config.js: server-side rewrites so the standalone frontend (built with NEXT_PUBLIC_API_URL=/deepflow-backend) can reach deepflow-backend:8000/api/* without CORS; target overridable via DEEPFLOW_BACKEND_HOST - docker-compose: expose deepflow-frontend 3001:3000 and deepflow-backend 8000:8000	2026-08-31 06:23:31 +08:00
figmar	25d46e8a9e	fix(deepflow-frontend): pin pnpm@10.6.5 to match packageManager field ... 'npm install -g pnpm' pulled pnpm 11.x, whose engine-identity verification fails against the 10.6.5 lockfile (ERR_PNPM_PNPM_ENGINE_IDENTITY_UNVERIFIABLE). Pin the exact version in both deps and builder stages, and use --frozen-lockfile for the install.	2026-08-31 05:53:02 +08:00
figmar	99cdc24e44	fix(images): vendor spacy zh model + nltk data for air-gapped GitHub builds ... - runtime: install zh_core_web_sm from vendored wheel (reused from runtime-ray) instead of 'python -m spacy download' (GitHub 403 on server) - runtime / runtime-ray / backend-python: replace nltk.download (GitHub + nltk_data.nltk.org unreachable from server) with vendored zips extracted to /usr/local/nltk_data/packages/{tokenizers,taggers}	2026-08-31 05:09:05 +08:00
figmar	7fcef906de	fix(gateway): Spring 6.2 + Base64 API migrations (ResponseCookie, statusCode, encodeToString, orElseThrow, effectively-final) ... - CasdoorOauthController: stateCookie -> ResponseCookie (HttpCookie.builder removed in 6.2), B64URL encoder/decoder split, encodeBytes -> encodeToString - CasdoorClient: resp.status() -> resp.statusCode() - CasdoorIdTokenVerifier: Optional.orElseThrow in map (type witness), no ternary+Mono.error - CasdoorUserService: throw instead of Mono.error inside fromCallable (type inference) - AuthFilter: effectively-final local for lambda capture	2026-08-31 04:15:58 +08:00
figmar	9b8f96c877	fix(gateway): postToken param type BodyInserter<?, ? super ClientHttpRequest> (matches body() signature)	2026-08-31 03:28:22 +08:00
figmar	b1caabdd18	fix(gateway): use BodyInserter supertype — Spring 6.2 renamed FormDataSpec to FormInserter	2026-08-31 03:25:53 +08:00
figmar	6cae8d4988	fix(gateway): kid extraction via Header map cast (0.12 has no typed get)	2026-08-31 03:10:35 +08:00
figmar	193aad5836	fix(security-common): Claims setters removed in jjwt 0.12; rebuild claims via JwtBuilder	2026-08-31 03:06:11 +08:00
figmar	c19a5eea08	fix(gateway): migrate jjwt to 0.12.6 and fix removed Spring WebClient FormDataSpec ... - CasdoorIdTokenVerifier: RSAKey does not exist in any jjwt release; build RSAPublicKey from JWKS n/e via RSAPublicKeySpec, use 0.12 parser API - JwtUtils / UserService: parserBuilder/parseClaimsJws -> parser()/parseSignedClaims - CasdoorClient: WebClient.RequestBodySpec.FormDataSpec removed in Spring 6.2, use BodyInserters.FormDataSpec	2026-08-31 02:52:09 +08:00
figmar	b94dae79fe	deepdata: fix main application class filename (rebrand residue broke build)	2026-08-31 02:22:01 +08:00
figmar	173e2866a3	deepdata: track product config and env defaults for self-contained deployment	2026-08-30 19:27:22 +08:00
figmar	e1d21036f8	deepdata: product docs, rebranded quick start, security policy	2026-08-30 19:23:42 +08:00
figmar	7bbc9d4ef0	deepdata: delivery stack (docker compose, helm, CI, build scripts)	2026-08-30 19:23:42 +08:00
figmar	bd3db30be9	deepdata: core product code (backend, frontend, runtime, deepflow module)	2026-08-30 19:23:38 +08:00