Commit 7a0725a3 authored by Jan Reimes's avatar Jan Reimes
Browse files

feat(ai): migrate from tdoc-ai to 3gpp-ai package 

* Update dependencies in pyproject.toml to replace tdoc-ai with 3gpp-ai.
* Refactor ai_app.py to import and utilize 3gpp-ai functionalities.
* Modify app.py to check for 3gpp-ai package availability.
* Adjust args.py to include new options for AI commands.
parent 7c036fb2

.config/mise/config.toml

+1 −1
Original line number Diff line number Diff line
@@ -26,7 +26,7 @@ run = ["mise install", "mise up", "mise prune"] 


[tasks.env-sync]

description = "Sync the virtual environment with pyproject.toml dependencies"

run = "uv sync -U --dev --all-extras --all-groups"

run = "uv sync -U --dev --all-extras --all-groups --all-packages"



[tasks.format]

description = "Format source code and tests"

AGENTS.md

+3 −3
Original line number Diff line number Diff line
@@ -35,7 +35,7 @@ Notes: 
|-----------|--------------|

| Core | Python 3.14, typer, rich, pydantic, pydantic-sqlite, requests, hishel |

| Specs Crawling | beautifulsoup4, lxml, xlsxwriter, zipinspect |

| AI Module | tdoc-ai (LanceDB, sentence-transformers, Docling, litellm) |

| AI Module | 3gpp-ai (LanceDB, sentence-transformers, Docling, litellm) |

| Conversion | convert-lo (LibreOffice headless) |

| Database | SQLite via pydantic-sqlite |
@@ -49,7 +49,7 @@ Notes: 
| `src/tdoc_crawler/specs/` | Specification operations |

| `src/tdoc_crawler/meetings/` | Meeting data handling |

| `src/tdoc_crawler/parsers/` | Parsing logic (Excel, HTML, etc.) |

| `packages/tdoc-ai/` | AI embeddings, graphs, search |

| `packages/3gpp-ai/` | AI embeddings, graphs, search |

| `packages/convert-lo/` | LibreOffice document conversion |

| `packages/pool-executors/` | Serial/parallel executor utilities |

| `tests/` | Test suite (see tests/AGENTS.md) |
@@ -115,7 +115,7 @@ Notes: 
|-----------|---------|

| `src/tdoc_crawler/` | Import patterns, HTTP caching, anti-duplication |

| `src/tdoc_crawler/cli/` | CLI patterns |

| `packages/tdoc-ai/` | AI pipeline, embeddings, graph patterns |

| `packages/3gpp-ai/` | AI pipeline, embeddings, graph patterns |

| `packages/convert-lo/` | LibreOffice conversion |

| `tests/` | Testing patterns, fixtures, coverage |

README.md

+1 −1
Original line number Diff line number Diff line
@@ -36,7 +36,7 @@ uv add 3gpp-crawler 
# Install with AI features (optional)

uv add 3gpp-crawler[ai]



# AI features are provided by the optional `tdoc-ai` extension package

# AI features are provided by the optional `3gpp-ai` extension package

# and installed automatically via the extra above.



# Or install from source

docs/ai.md

+1 −1
Original line number Diff line number Diff line
@@ -41,7 +41,7 @@ uv sync --extra ai 


All required dependencies (Kreuzberg, LiteLLM, sentence-transformers, LanceDB) are installed automatically.



Internally, AI capabilities are provided by the optional `tdoc-ai` package, which is pulled in by `3gpp-crawler[ai]`.

Internally, AI capabilities are provided by the optional `3gpp-ai` package, which is pulled in by `3gpp-crawler[ai]`.



______________________________________________________________________

docs/history/2026-03-06_SUMMARY_01_AI_EMBEDDINGS_ACCELERATE_BACKEND.md

+1 −1
Original line number Diff line number Diff line
@@ -20,7 +20,7 @@ Added an `--accelerate` option for `ai process` so callers can select the senten 


### Dependencies



- Updated `tdoc-ai` dependencies to ensure all requested backends can be installed.

- Updated `3gpp-ai` dependencies to ensure all requested backends can be installed.



## Verification