Commit dda1999c authored by Jan Reimes's avatar Jan Reimes
Browse files

feat(tasks-template): Update task list template for feature implementation

- Change description format to markdown header.
- Clarify domain placement requirements for HTTP calls.
- Add explicit exemption documentation for AI model-provider endpoints.
- Improve organization and formatting for better readability.
- Ensure consistent use of escape characters in test paths.

feat(agents): Refine HTTP client guidelines for caching

- Specify that core crawler source traffic must use `create_cached_session()`.
- Exempt AI model-provider traffic from caching requirements.
- Emphasize the importance of reducing unnecessary traffic to 3GPP servers.

feat(ai-document-processing): Enhance AI feature specifications and tasks

- Update status to "Implementation Ready".
- Add clarifications regarding AI configuration variables and linter rules.
- Document approved provider/client integrations for AI model-provider traffic.
- Refine user scenarios and testing requirements for clarity.
- Ensure all HTTP requests for core crawler sources use `create_cached_session()`.

feat(ai-document-processing): Revise tasks for AI feature implementation

- Update task descriptions for clarity and consistency.
- Ensure all tasks are aligned with the new specifications and guidelines.
- Add checkpoints for user approval and red-phase validation.
- Clarify dependencies and execution order for user stories.
parent 9ed2ab96
Loading
Loading
Loading
Loading
+28 −17
Original line number Diff line number Diff line
<!--
Sync Impact Report:
- Version change: 1.2.0 -> 1.3.0
- Modified principles: None
- Added sections: Core Principles V, VI, VII; Implementation Gates 5, 6, 7
- Version change: 1.3.0 -> 2.0.0
- Modified principles: VII Responsible Network Access -> VII Responsible Network Access (Core Crawler Scope)
- Added sections: None
- Removed sections: None
- Templates requiring updates:
  - UPDATED: .specify/templates/plan-template.md (Constitution Check section)
  - UPDATED: .specify/templates/spec-template.md (FR-010, FR-011, FR-012)
  - UPDATED: .specify/templates/tasks-template.md (domain placement reminder)
  - ✅ updated: .specify/templates/plan-template.md
  - ✅ updated: .specify/templates/spec-template.md
  - ✅ updated: .specify/templates/tasks-template.md
  - N/A: .specify/templates/commands/*.md (directory not present)
  - ✅ updated: AGENTS.md
- Follow-up TODOs: None
-->

# tdoc-crawler Constitution

## Core Principles

### I. Library-First Development

Every feature must start as a standalone, reusable library component. Application or CLI
code may only call into this library; direct feature implementation inside application
modules is prohibited. Libraries must be independently testable and have a clear,
reusable interface.

### II. CLI Text + JSON Interface

Every library must expose its functionality through a command-line interface. Each CLI
must accept text input via stdin, arguments, or files, and must emit text output via
stdout. Every CLI must support a JSON output mode for structured data exchange; errors
@@ -31,6 +35,7 @@ Exception: For the `crawl-specs`, `query-specs`, `checkout-spec`, and `open-spec
commands, JSON output is optional for the initial release of the specs feature.

### III. Test-Driven Development (Non-Negotiable)

Implementation code must not be written until unit tests are written, reviewed by the
user, and explicitly approved. The approved tests must be executed and verified to fail
(red phase) before any implementation begins. Development must follow the
@@ -75,16 +80,21 @@ domain logic MUST NOT be copied into `cli/`, test helpers, or utility modules.
Rationale: code duplication was the primary driver of the domain-oriented refactoring.
Duplicated logic makes bugs harder to find and fixes harder to apply consistently.

### VII. Responsible Network Access
### VII. Responsible Network Access (Core Crawler Scope)

For the core `tdoc-crawler` source-fetching workflow (3gpp.org, whatthespec.net, portal
resources), all HTTP requests MUST use `create_cached_session()` from
`tdoc_crawler.http_client` to enable hishel caching. Direct HTTP requests to these
sources are prohibited.

All HTTP requests MUST use `create_cached_session()` from `tdoc_crawler.http_client` to
enable hishel caching. Direct HTTP requests to external services (3gpp.org,
whatthespec.net) are prohibited. Features that introduce network calls MUST plan and
document their use of the caching layer before implementation begins.
This principle does NOT apply to AI model-provider traffic used by the AI feature
(for example, LLM or embedding provider endpoints). AI features MUST still document
their network access approach in spec/plan artifacts and MUST NOT bypass approved
provider/client integrations.

Rationale: the 3GPP portal and related services have strict rate-limiting policies. The
caching layer reduces traffic by 50–90% on incremental crawls and protects against
blocking.
Rationale: core crawler sources have strict rate-limiting policies where caching reduces
traffic by 50–90% on incremental crawls. AI provider traffic is non-deterministic and
requires a separate integration policy.

## Implementation Gates

@@ -99,8 +109,9 @@ blocking.
  `cli/` or `utils/`; no reference to the legacy `crawlers/` package.
- Gate 6: Existing functionality has been searched before implementation; no duplicated
  logic is introduced; CLI delegates to core library.
- Gate 7: All HTTP requests use `create_cached_session()`; no direct network calls to
  external services.
- Gate 7: Core crawler HTTP requests use `create_cached_session()`; no direct network
  calls to 3GPP/WhatTheSpec sources. AI model-provider traffic is exempt but must be
  explicitly documented in feature specs/plans.

## Development Workflow

@@ -119,4 +130,4 @@ blocking.
- Compliance is verified in specs, plans, task lists, and code reviews; non-compliant
  changes must be blocked or accompanied by an approved amendment.

**Version**: 1.3.0 | **Ratified**: 2026-02-05 | **Last Amended**: 2026-02-24
**Version**: 2.0.0 | **Ratified**: 2026-02-05 | **Last Amended**: 2026-02-24
+1 −0
Original line number Diff line number Diff line
@@ -25,4 +25,5 @@ Auto-generated from all feature plans. Last updated: [DATE]
[LAST 3 FEATURES AND WHAT THEY ADDED]

<!-- MANUAL ADDITIONS START -->

<!-- MANUAL ADDITIONS END -->
+11 −9
Original line number Diff line number Diff line
@@ -24,7 +24,7 @@
**Target Platform**: [e.g., Linux server, iOS 15+, WASM or NEEDS CLARIFICATION]
**Project Type**: [single/web/mobile - determines source structure]
**Performance Goals**: [domain-specific, e.g., 1000 req/s, 10k lines/sec, 60 fps or NEEDS CLARIFICATION]
**Constraints**: [domain-specific, e.g., <200ms p95, <100MB memory, offline-capable or NEEDS CLARIFICATION]
**Constraints**: [domain-specific, e.g., \<200ms p95, \<100MB memory, offline-capable or NEEDS CLARIFICATION]
**Scale/Scope**: [domain-specific, e.g., 10k users, 1M LOC, 50 screens or NEEDS CLARIFICATION]

## Constitution Check
@@ -39,18 +39,19 @@
- [ ] Domain placement confirmed (correct domain package; no logic in cli/ or utils/;
  no legacy crawlers/ reference).
- [ ] DRY validation done (existing code searched; no duplicated logic introduced).
- [ ] Network access plan documented (all HTTP via create_cached_session(); no direct
  calls to external services).
- [ ] Network access plan documented (core crawler source traffic via
  create_cached_session(); any AI model-provider traffic explicitly documented as
  exempt with approved provider/client integration).

## Library Boundary & CLI Contract

[Describe the standalone library module, its public API, and the CLI entrypoints.
Include text input/output channels and JSON output mode.]
\[Describe the standalone library module, its public API, and the CLI entrypoints.
Include text input/output channels and JSON output mode.\]

## TDD Evidence

[Describe the unit tests to be written first and how failing tests (red phase) will be
validated before implementation begins.]
\[Describe the unit tests to be written first and how failing tests (red phase) will be
validated before implementation begins.\]

## Project Structure

@@ -67,6 +68,7 @@ specs/[###-feature]/
```

### Source Code (repository root)

<!--
  ACTION REQUIRED: Replace the placeholder tree below with the concrete layout
  for this feature. Delete unused options and expand the chosen structure with
@@ -110,8 +112,8 @@ ios/ or android/
└── [platform-specific structure: feature modules, UI flows, platform tests]
```

**Structure Decision**: [Document the selected structure and reference the real
directories captured above]
**Structure Decision**: \[Document the selected structure and reference the real
directories captured above\]

## Complexity Tracking

+10 −8
Original line number Diff line number Diff line
@@ -26,14 +26,14 @@

**Why this priority**: [Explain the value and why it has this priority level]

**Independent Test**: [Describe how this can be tested independently - e.g., "Can be fully tested by [specific action] and delivers [specific value]"]
**Independent Test**: \[Describe how this can be tested independently - e.g., "Can be fully tested by [specific action] and delivers [specific value]"\]

**Acceptance Scenarios**:

1. **Given** [initial state], **When** [action], **Then** [expected outcome]
2. **Given** [initial state], **When** [action], **Then** [expected outcome]
1. **Given** [initial state], **When** [action], **Then** [expected outcome]

---
______________________________________________________________________

### User Story 2 - [Brief Title] (Priority: P2)

@@ -47,7 +47,7 @@

1. **Given** [initial state], **When** [action], **Then** [expected outcome]

---
______________________________________________________________________

### User Story 3 - [Brief Title] (Priority: P3)

@@ -61,7 +61,7 @@

1. **Given** [initial state], **When** [action], **Then** [expected outcome]

---
______________________________________________________________________

[Add more user stories as needed, each with an assigned priority]

@@ -105,8 +105,10 @@
- **FR-011**: Before implementing any new functionality, existing code MUST be searched
  for equivalent implementations; CLI code MUST only delegate to core library functions
  and MUST NOT duplicate domain logic.
- **FR-012**: All HTTP requests MUST use `create_cached_session()` from
  `tdoc_crawler.http_client`; direct HTTP calls to external services are prohibited.
- **FR-012**: For core crawler source traffic (3GPP/WhatTheSpec/portal), all HTTP
  requests MUST use `create_cached_session()` from `tdoc_crawler.http_client`; direct
  HTTP calls to those sources are prohibited. AI model-provider traffic MAY be exempt,
  but the feature MUST explicitly document the exemption and approved client path.

*Example of marking unclear requirements:*

@@ -130,4 +132,4 @@
- **SC-001**: [Measurable metric, e.g., "Users can complete account creation in under 2 minutes"]
- **SC-002**: [Measurable metric, e.g., "System handles 1000 concurrent users without degradation"]
- **SC-003**: [User satisfaction metric, e.g., "90% of users successfully complete primary task on first attempt"]
- **SC-004**: [Business metric, e.g., "Reduce support tickets related to [X] by 50%"]
- **SC-004**: \[Business metric, e.g., "Reduce support tickets related to [X] by 50%"\]
+33 −32
Original line number Diff line number Diff line
---
______________________________________________________________________

description: "Task list template for feature implementation"
---
## description: "Task list template for feature implementation"

# Tasks: [FEATURE NAME]

@@ -12,9 +11,11 @@ description: "Task list template for feature implementation"
exempted by the feature specification and approved in the constitution.

**Domain placement**: Every implementation task MUST place code in the correct domain
package (`tdocs/`, `meetings/`, `specs/`). Tasks that add HTTP calls MUST include a
sub-task verifying use of `create_cached_session()`. Search for existing implementations
before creating new files.
package (`tdocs/`, `meetings/`, `specs/`). Tasks that add core crawler source HTTP calls
MUST include a sub-task verifying use of `create_cached_session()`. If a feature uses
AI model-provider endpoints, tasks MUST include explicit exemption documentation and the
approved provider/client integration path. Search for existing implementations before
creating new files.

**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story.

@@ -58,7 +59,7 @@ before creating new files.
- [ ] T002 Initialize [language] project with [framework] dependencies
- [ ] T003 [P] Configure linting, formatting, and type checking (Ruff, Ty)

---
______________________________________________________________________

## Phase 2: Foundational (Blocking Prerequisites)

@@ -77,7 +78,7 @@ Examples of foundational tasks (adjust based on your project):

**Checkpoint**: Foundation ready - user story implementation can now begin in parallel

---
______________________________________________________________________

## Phase 3: User Story 1 - [Title] (Priority: P1) 🎯 MVP

@@ -89,8 +90,8 @@ Examples of foundational tasks (adjust based on your project):

> **NOTE: Write these tests FIRST, ensure they FAIL before implementation**

- [ ] T010 [P] [US1] Contract test for [endpoint] in tests/contract/test_[name].py
- [ ] T011 [P] [US1] Integration test for [user journey] in tests/integration/test_[name].py
- [ ] T010 [P] [US1] Contract test for [endpoint] in tests/contract/test\_[name].py
- [ ] T011 [P] [US1] Integration test for [user journey] in tests/integration/test\_[name].py

### Implementation for User Story 1

@@ -103,7 +104,7 @@ Examples of foundational tasks (adjust based on your project):

**Checkpoint**: At this point, User Story 1 should be fully functional and testable independently

---
______________________________________________________________________

## Phase 4: User Story 2 - [Title] (Priority: P2)

@@ -113,8 +114,8 @@ Examples of foundational tasks (adjust based on your project):

### Tests for User Story 2 (REQUIRED) ⚠️

- [ ] T018 [P] [US2] Contract test for [endpoint] in tests/contract/test_[name].py
- [ ] T019 [P] [US2] Integration test for [user journey] in tests/integration/test_[name].py
- [ ] T018 [P] [US2] Contract test for [endpoint] in tests/contract/test\_[name].py
- [ ] T019 [P] [US2] Integration test for [user journey] in tests/integration/test\_[name].py

### Implementation for User Story 2

@@ -125,7 +126,7 @@ Examples of foundational tasks (adjust based on your project):

**Checkpoint**: At this point, User Stories 1 AND 2 should both work independently

---
______________________________________________________________________

## Phase 5: User Story 3 - [Title] (Priority: P3)

@@ -135,8 +136,8 @@ Examples of foundational tasks (adjust based on your project):

### Tests for User Story 3 (REQUIRED) ⚠️

- [ ] T024 [P] [US3] Contract test for [endpoint] in tests/contract/test_[name].py
- [ ] T025 [P] [US3] Integration test for [user journey] in tests/integration/test_[name].py
- [ ] T024 [P] [US3] Contract test for [endpoint] in tests/contract/test\_[name].py
- [ ] T025 [P] [US3] Integration test for [user journey] in tests/integration/test\_[name].py

### Implementation for User Story 3

@@ -146,11 +147,11 @@ Examples of foundational tasks (adjust based on your project):

**Checkpoint**: All user stories should now be independently functional

---
______________________________________________________________________

[Add more user story phases as needed, following the same pattern]

---
______________________________________________________________________

## Phase N: Polish & Cross-Cutting Concerns

@@ -164,7 +165,7 @@ Examples of foundational tasks (adjust based on your project):
- [ ] TXXX Security hardening
- [ ] TXXX Run quickstart.md validation

---
______________________________________________________________________

## Dependencies & Execution Order

@@ -200,7 +201,7 @@ Examples of foundational tasks (adjust based on your project):
- Models within a story marked [P] can run in parallel
- Different user stories can be worked on in parallel by different team members

---
______________________________________________________________________

## Parallel Example: User Story 1

@@ -214,38 +215,38 @@ Task: "Create [Entity1] model in src/models/[entity1].py"
Task: "Create [Entity2] model in src/models/[entity2].py"
```

---
______________________________________________________________________

## Implementation Strategy

### MVP First (User Story 1 Only)

1. Complete Phase 1: Setup
2. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
3. Complete Phase 3: User Story 1
4. **STOP and VALIDATE**: Test User Story 1 independently
5. Deploy/demo if ready
1. Complete Phase 2: Foundational (CRITICAL - blocks all stories)
1. Complete Phase 3: User Story 1
1. **STOP and VALIDATE**: Test User Story 1 independently
1. Deploy/demo if ready

### Incremental Delivery

1. Complete Setup + Foundational → Foundation ready
2. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
3. Add User Story 2 → Test independently → Deploy/Demo
4. Add User Story 3 → Test independently → Deploy/Demo
5. Each story adds value without breaking previous stories
1. Add User Story 1 → Test independently → Deploy/Demo (MVP!)
1. Add User Story 2 → Test independently → Deploy/Demo
1. Add User Story 3 → Test independently → Deploy/Demo
1. Each story adds value without breaking previous stories

### Parallel Team Strategy

With multiple developers:

1. Team completes Setup + Foundational together
2. Once Foundational is done:
1. Once Foundational is done:
   - Developer A: User Story 1
   - Developer B: User Story 2
   - Developer C: User Story 3
3. Stories complete and integrate independently
1. Stories complete and integrate independently

---
______________________________________________________________________

## Notes

Loading