Commit db35f206 authored by Jan Reimes's avatar Jan Reimes
Browse files
# Conflicts:
#	AGENTS.md
#	docs/REVIEW_AND_IMPROVEMENTS_AGENTS_MD.md
#	pyproject.toml
#	src/tdoc_crawler/cli/app.py
#	src/tdoc_crawler/cli/helpers.py
#	src/tdoc_crawler/crawlers/meetings.py
#	src/tdoc_crawler/crawlers/parallel.py
#	src/tdoc_crawler/crawlers/portal.py
#	src/tdoc_crawler/database/connection.py
#	src/tdoc_crawler/models/__init__.py
#	src/tdoc_crawler/models/meetings.py
#	src/tdoc_crawler/models/tdocs.py
parents d910deb0 3fbff5e2
Loading
Loading
Loading
Loading
+138 −0
Original line number Diff line number Diff line
---
name: 3gpp-basics
description: General 3GPP organization overview, partnerships, scope, and fundamental concepts. Use when working with 3GPP data structures, understanding 3GPP hierarchy, or needing context about 3GPP as an organization.
related: 
  - 3gpp-working-groups
  - 3gpp-meetings
  - 3gpp-tdocs
  - 3gpp-portal-authentication
---

# 3GPP Basics

## Overview

3GPP (3rd Generation Partnership Project) is a collaboration between seven telecommunications standard development organizations (Organizational Partners) providing their members with a stable environment to produce Reports and Specifications that define 3GPP technologies.

## Organizational Partners

The seven Organizational Partners are:

- **ARIB** (Association of Radio Industries and Businesses)
- **ATIS** (Alliance for Telecommunications Industry Solutions)
- **CCSA** (China Communications Standards Association)
- **ETSI** (European Telecommunications Standards Institute)
- **TSDSI** (Telecommunications Standards Association of India)
- **TTA** (Telecommunications Technology Association)
- **TTC** (Telecommunication Technology Committee)

## Technical Administration

Since 3GPP is not a legal entity, technical administration and infrastructure is provided by ETSI. This includes:

- Meeting information and registration: <https://portal.etsi.org/Meetings.aspx#/>
- Portal access for certain resources
- File servers and infrastructure

### MCC (Mobile Competence Centre)

The MCC is the technical support team for 3GPP operations:

- Maintains 3GPP specifications and FTP servers
- Publishes new/revised specs after each TSG round
- Manages the CR database and work item tracking
- Provides meeting document templates
- Target: specs available within 3-4 weeks after TSG plenary

### PCG (Project Coordination Group)

The PCG coordinates the 3GPP partnership:

- Coordinates work across the three TSGs
- Manages relationships with Organizational Partners
- Oversees 3GPP budget and resources
- Strategic planning for 3GPP direction

## Technical Specification Groups (TSGs)

3GPP has three main Technical Specification Groups (TSGs):

### TSG RAN (Radio Access Network)

- **Focus**: Radio aspects of mobile communications
- **Responsibilities**: Radio access technologies like LTE and 5G NR
- **FTP Root**: `https://www.3gpp.org/ftp/tsg_ran/`

### TSG SA (Service and System Aspects)

- **Focus**: Overall architecture and service aspects
- **Responsibilities**: Core network functionalities, service requirements
- **FTP Root**: `https://www.3gpp.org/ftp/tsg_sa/`

### TSG CT (Core Network and Terminals)

- **Focus**: Core network and terminal aspects
- **Responsibilities**: Protocols, interfaces between core network and user equipment
- **FTP Root**: `https://www.3gpp.org/ftp/tsg_ct/`

## Scope

3GPP specifications cover cellular telecommunications technologies, including:

- Radio access
- Core network
- Service capabilities
- Hooks for non-radio access to core network
- Interworking with non-3GPP networks

## Specifications and Work

3GPP specifications and studies (TRs) are contribution-driven by member companies in Working Groups and at Technical Specification Group (TSG) level.

The Working Groups within TSGs meet regularly and come together for their quarterly TSG Plenary meetings, where their work is presented for information, discussion, and approval.

## Releases and Generational Approach

3GPP technologies evolve through generations of commercial cellular/mobile systems. While "generations" (3G, 4G, 5G) serve as adequate descriptors for the type of network under discussion, real progress is measured by achievements in particular **Releases**.

### Key Release Concepts

- **Release Freeze**: New features are "functionally frozen" when a Release is completed
- **Backward Compatibility**: Major focus is to ensure systems are backwards and forwards compatible where possible
- **Parallel Development**: 3GPP works on multiple Releases in parallel, starting future work well in advance
- **Current Releases**: Release 20 (current), Release 19, Release 18, etc. down to Release 1999

### Example: Dual Connectivity for 5G

Many operators use dual connectivity between LTE and 5G NR equipment, using "Non-Standalone" work completed early in Release 15. Forward compatibility was built into Non-Standalone NR equipment to ensure it works on Standalone 5G NR systems.

## TDocs (Temporary Documents)

TDocs are meeting documents produced by members participating in 3GPP Working Groups and Sub-Working Groups. They include:

- Proposals for new features
- Reports on technical work
- Other documents related to standard development

Every TDoc is always allocated to a specific 3GPP meeting.

## Portal Authentication

To access certain 3GPP resources, you need an ETSI Online Account (EOL). However:

- Files on the 3GPP FTP/HTTP server are publicly accessible
- For parsing metadata or accessing certain webpages, login may be required

## See Also

- **@3gpp-working-groups** - Detailed information about Working Groups, Sub-Working Groups, tbid/SubTB identifiers
- **@3gpp-meetings** - Meeting structure, naming conventions, and web pages
- **@3gpp-tdocs** - TDoc patterns, metadata, and file server access
- **@3gpp-portal-authentication** - EOL authentication, AJAX login patterns

## Resources

- 3GPP Official Website: <https://www.3gpp.org/>
- 3GPP Work Plan: <https://www.3gpp.org/specifications-technologies/3gpp-work-plan/>
- 3GPP Portal: <https://portal.3gpp.org/>
- ETSI Portal: <https://portal.etsi.org/>
+290 −0
Original line number Diff line number Diff line
---
name: 3gpp-change-request
description: Change Request procedure, workflow, status tracking, CR database, and step-by-step instructions. Use when working with 3GPP Change Requests to modify specifications, understanding CR lifecycle, or querying CR status.
related:
  - 3gpp-specifications
  - 3gpp-basics
  - 3gpp-working-groups
  - 3gpp-tdocs
  - 3gpp-portal-authentication
  - 3gpp-releases
---

# 3GPP Change Requests - Reference

> **Quick Start**: For workflow overview and submission checklist, see [workflow.md](workflow.md).

## Overview

Change Requests (CRs) are documents submitted to 3GPP Working Groups to create revised versions of specifications. They represent the formal mechanism for modifying 3GPP specs after initial approval.

## What is a CR?

A CR is a **temporary document** (tdoc) to a meeting which specifies in precise detail changes to be made to a specification. It consists of:

- **CR Cover Sheet**: Describes why a change is needed and summarizes how the change is made
- **Attached Parts of Spec**: Word™ "Track Changes" (revision marks) identifying affected sections
- **Summary of Changes**: Brief explanation of the proposed modification

## Why Changes Are Needed

Three main reasons for creating a CR:

1. **Add a new feature** to a Release still under development
2. **Correct / clarify / enhance** an existing feature in a Release still under development
3. **Correct an error** in a spec which is functionally frozen

## CR Development and Approval

### Submission Process

1. **WG Level**: Any 3GPP member organization can propose a CR to the Working Group responsible for the spec
2. **WG Agreement**: WG discusses and agrees CR is valid and required
3. **TSG Approval**: WG presents CR to parent TSG plenary for final approval
4. **MCC Incorporation**: After TSG approval, 3GPP Support Team (MCC) incorporates CR into new spec version

### CR Status Meanings

| Status | TSG | WG | Meaning |
|---------|-----|----|--------|
| use | YES | YES | Valid for use |
| agreed | - | YES | WG agreed, forward to TSG |
| approved | YES | NO | Final decision (implemented) |
| rejected | YES | YES | Sustained objection |
| revised | YES | YES | Modified to new revision |
| merged | YES | YES | Combined with other CRs |
| postponed | YES | YES | Deferred to later date |
| endorsed | NO | YES | WG consensus, technically correct |
| withdrawn | YES | YES | Retracted before discussion |
| reissued | NO | NO | Recast unchanged in another TSG |
| noted | NO | NO | Deprecated (ambiguous term) |

## CR Number Format

```
<specnumber_no_dot>*_CR*<4-character_CR_number>*[r*<revision_number>*]
```

**Components:**

- `<specnumber_no_dot>`: Spec number without dot (e.g., `21.456` for TS 21.456)
- `*`: Literal asterisk
- `<4-character_CR_number>`: 4-digit CR number padded with leading zeros (e.g., `0095`)
- `[r*<revision_number>*]`: Lowercase 'r' + revision number (only for revised CRs)
- `<release>`: Release identifier (Rel-4, Rel-5, etc.)

**Examples:**

- Initial: `31.102_0095` - CR 0095 to spec 31.102
- First revision: `31.102_0095_r1` - Revision 1 of spec 31.102
- Mirror CR: `31.103_0095` - Mirrored CR to spec 31.103

## CR Database

### Netovate Search Tool

3GPP has an agreement with Netovate to provide a free CR database search:

- **URL**: <http://netovate.com/cr-search/>
- **Access**: Free web-based search tool
- **Features**: Search by spec number, CR number, or status

### 3GPP CR Database

Available on 3GPP FTP:

- **Directory**: `/ftp/Information/Databases/Change_Request/`
- **Format**: MS Access™ (.mdb) database
- **Access**: Direct download or via portal

### CR Status Tracking

Each spec maintains a history box listing all CRs that have been approved for that specification.

## Step-by-Step Instructions

3GPP provides detailed step-by-step instructions for writing CRs:

- **URL**: <https://www.3gpp.org/specifications-technologies/specifications-by-series/change-requests-step-by-step>

**Key Sections:**

1. Header (meeting details, WG document number)
2. Spec number, CR number
3. CR revision number (for revised CRs only)
4. Current version
5. Title
6. Source to WG
7. Source to TSG
8. Work item code
9. Date
10. Category
11. Release
12. Consequences if not approved
13. Clauses affected
14. Other specs affected
15. Other comments
16. CR revision history
17. Filename convention
18. Body of CR
19. Other considerations

## CR Categories

CRs are classified by category to indicate their nature:

- **Category F (Corrective)**: Corrective CRs, typically to earliest Release version
- **Category A (Mirror)**: Mirrors of CRs to previous Release versions
- **Mirror CRs require**: Same WI code per Release
- **Mirror CRs are marked**: Category A in spec history

### Category Definitions

| Category | Description |
|-----------|-------------|
| F | Corrective CRs applied to earliest Release version |
| A | Mirror CRs of same spec to different Release version |
| TEI | Technical Enhancement or Improvement |

## Important CR Concepts

### Mirror CRs

For specs maintained in multiple parallel Releases:

- Each Release needs its own mirror CR
- Use same WI code (Work Item code) for all mirror CRs
- Example: Spec 21.456 has mirror CRs in Release 4, Release 5, etc.

### Corrective Release 7 CRs

Special category for early LTE (Release 7) features that need to work on both Release 4 and Release 5 specs.

### Editorial Updates

CRs for pure editorial changes (no technical content) are typically implemented by MCC as "dot-one" versions (e.g., 21.456.1) rather than full CRs.

## Work Items

Each CR must be associated with a **Work Item (WI) code**:

- Obtained from official WI list: <https://www.3gpp.org/ftp/Specs/html-info/WI-List.htm>
- **Format**: 6-digit numeric code (e.g., `123456`)
- **Purpose**: Justifies the new feature or modification

## CR Submission Checklist

1. [ ] Unique_ID value from Work Item
2. [ ] TDoc number of Work Item Description (WID) document
3. [ ] Name of rapporteur for TS/TR (or contact coords for TR)
4. [ ] Target date from WID (approval date)
5. [ ] Source to WG (responsible 3GPP member)
6. [ ] Source to TSG (if presented directly to TSG)
7. [ ] Work item code (from WI list)
8. [ ] Release (one only)
9. [ ] Category (TEI for technical enhancements)
10. [ ] Title (descriptive, not redundant)
11. [ ] Reason for change
12. [ ] Summary of changes
13. [ ] Consequences if not approved
14. [ ] Clauses affected (list individually)
15. [ ] Other specs affected
16. [ ] Other comments (optional)
17. [ ] Date (format: yyyy-MM-dd)
18. [ ] Release (Rel-4, Rel-5, etc.)
19. [ ] (U)SIM - ME/UE - Radio Access Network - Core Network checkboxes
20. [ ] UICC field (change to (U)SIM or ISIM)

### CR Revision History

Optional field in CR document to track changes as CR passes through revisions:

- Revision 1: Initial CR
- Revision 2: First revision of CR
- Revision 3: Subsequent revisions

### File Naming Convention

**Word™ files for CRs** follow 3GPP naming:

```
*<specnumber_no_dot>*_CR*<4-character_CR_number>*[r*<revision_number>*]
```

**Zip™ archive** containing CRs:

- Word™ files (CR cover sheet + body + attachments)
- Zip™ file named according to convention
- Contains multiple CRs packaged together for a TSG meeting

### CR Lifecycle

```
Draft → WG Agreement → TSG Approval → MCC Incorporation → Published Spec
```

## Change Request Workflow

### 1. Draft CR

Member organization creates CR document


### 2. Submit to WG

WG discusses and agrees CR is valid


### 3. WG Presents to TSG

WG presents CR at TSG plenary for approval


### 4. TSG Approval

TSG approves CR (becomes decision)


### 5. MCC Incorporates

MCC incorporates CR into new spec version


### 6. Publish

New spec version becomes available

## Common Pitfalls

1. **Multiple Releases**: One spec may be maintained in multiple Releases
2. **Parallel CRs**: Need separate CRs for each Release with same WI code
3. **Editorial Changes**: Use "dot-one" versions, not full CRs
4. **Wrong Release**: Can't use Release 6 WI code for Release 4 spec
5. **Out-of-date Specs**: Writing CR to old version that's been updated

## Best Practices

1. **Always use WI codes** from official list
2. **Check spec version** before writing CR (is it in Release 4, Release 5, or current?)
3. **Target correct Release**: Each CR should target only one Release (unless mirror CR)
4. **Use proper categories**: TEI for technical enhancements, F for correctives
5. **Mirror CR requirements**: Include mirror CRs for all affected Releases if maintaining parallel versions
6. **Document revisions**: Use CR revision history field for tracking changes
7. **Format dates correctly**: Use `yyyy-MM-DD` format

## Cross-References

- **@3gpp-basics** - 3GPP organization and structure
- **@3gpp-working-groups** - Working Group responsibility for specs
- **@3gpp-tdocs** - CRs as meeting documents (tdocs)
- **@3gpp-releases** - Understanding Release structure
- **@3gpp-portal-authentication** - Some CR documents may require EOL access

## Resources

- 3GPP Official: <https://www.3gpp.org/>
- 3GPP Portal: <https://portal.3gpp.org/>
- Step-by-step: <https://www.3gpp.org/specifications-technologies/specifications-by-series/change-requests-step-by-step>
- CR Database: /ftp/Information/Databases/Change_Request/
- WI List: <https://www.3gpp.org/ftp/Specs/html-info/WI-List.htm>
- Netovate: <http://netovate.com/cr-search/>
+95 −0
Original line number Diff line number Diff line
# Change Request Workflow

Quick reference for CR submission and approval process.

## CR Lifecycle

```text
Draft → WG Agreement → TSG Approval → MCC Incorporation → Published Spec
```

### 1. Draft CR

Member organization creates CR document containing:

- **CR Cover Sheet**: Why change is needed, summary of modifications
- **Attached Spec Sections**: Word "Track Changes" showing affected text
- **Summary of Changes**: Brief explanation

### 2. Submit to WG

- Any 3GPP member can propose CR to responsible Working Group
- WG discusses and agrees CR is valid and required

### 3. WG Presents to TSG

- WG presents CR at TSG plenary for final approval
- Status changes from "agreed" to pending TSG decision

### 4. TSG Approval

- TSG approves CR (status becomes "approved")
- CR becomes formal decision

### 5. MCC Incorporates

- 3GPP Support Team (MCC) incorporates CR into new spec version
- Target: 4 weeks after TSG plenary

### 6. Publish

- New spec version becomes available on FTP server

## Why CRs Are Needed

| Reason | When |
| ------ | ---- |
| Add new feature | Release still under development |
| Correct/clarify/enhance | Existing feature, Release still open |
| Fix error | Spec is functionally frozen |

## CR Categories

| Category | Description |
| -------- | ----------- |
| F | Corrective - applied to earliest Release version |
| A | Mirror - same CR to different Release version |
| TEI | Technical Enhancement or Improvement |

## CR Number Format

```text
<specnumber_no_dot>_CR<4-digit_number>[r<revision>]
```

**Examples:**

- `31102_CR0095` - CR 0095 to spec 31.102
- `31102_CR0095r1` - Revision 1 of that CR
- `31103_CR0095` - Mirror CR to spec 31.103

## Quick Checklist

Essential fields for CR submission:

- [ ] Work Item code (from [WI List](https://www.3gpp.org/ftp/Specs/html-info/WI-List.htm))
- [ ] Spec number and current version
- [ ] Target Release (one only)
- [ ] Category (F, A, or TEI)
- [ ] Source to WG
- [ ] Title and reason for change
- [ ] Clauses affected
- [ ] Date (yyyy-MM-dd format)

## Common Pitfalls

1. **Wrong Release**: Can't use Rel-6 WI code for Rel-4 spec
2. **Missing mirrors**: Need separate CR for each Release if spec is in multiple
3. **Out-of-date base**: Writing CR against old version that's been updated
4. **Wrong category**: Use TEI for enhancements, F for corrections

## See Also

- [SKILL.md](SKILL.md) - Full CR reference with status codes and detailed fields
- **@3gpp-specifications** - Spec numbering and structure
- **@3gpp-releases** - Release freeze concepts
+183 −0
Original line number Diff line number Diff line
---
name: 3gpp-meetings
description: Meeting structure, naming conventions, quarterly plenaries, and meeting pages. Use when parsing 3GPP meeting data, understanding meeting schedules, or working with meeting metadata from 3GPP portal.
related: 
  - 3gpp-basics
  - 3gpp-working-groups
  - 3gpp-tdocs
  - 3gpp-portal-authentication
---

# 3GPP Meetings

## Overview

3GPP Technical Specification Groups (TSGs) hold **quarterly plenary meetings** where their Working Groups present work for information, discussion, and approval. The three TSGs (RAN, SA, CT) meet synchronously over a one-week period.

### Meeting Schedule

**Plenary Period:** March, June, September, December each year

Working Groups meet **before** their TSG plenary meetings to make progress on specifications and studies that will be approved at the TSG level.

**Recent History:**

- Virtual meetings from early 2020 until June 2022 (due to COVID-19)
- Return to face-to-face meetings in June 2022
- Moving to hybrid format (F2F or virtual) since 2022
- Current approach: mix of F2F and virtual meetings

## Meeting Identifiers

### Meeting ID Format

Each 3GPP meeting has a unique integer `meeting_id` that serves as the primary database key.

### Meeting Name Format

Meeting names are **inconsistent** across years and groups. Do NOT parse information directly from meeting names.

**Examples:**

- `SA4#134` - Standard SA4 format
- `SA4-e (AH) Audio SWG post 130` - Ad-hoc meeting
- `3GPPSA4-e (AH) Audio SWG post 130` - With 3GPP prefix
- `S4-133-e` - Alternative format

**Important:** Meeting names cannot be reliably parsed. Always use the `meeting_id` integer from the portal for database operations.

## Meeting Pages

### Meeting List Pages

Dynamic pages generated by 3GPP showing all meetings for a group:

**URL Pattern:**

```
https://www.3gpp.org/dynareport?code=Meetings-<ID>.htm
```

**Meeting ID Codes:**

- `RP` - RAN Plenary meetings
- `SP` - SA Plenary meetings
- `CP` - CT Plenary meetings
- `R1` - RAN WG1 meetings
- `S4` - SA WG4 meetings
- `C1` - CT WG1 meetings

**Example URLs:**

- RAN meetings: `https://www.3gpp.org/dynareport?code=Meetings-RP.htm`
- SA4 meetings: `https://www.3gpp.org/dynareport?code=Meetings-S4.htm`
- CT3 meetings: `https://www.3gpp.org/dynareport?code=Meetings-C3.htm`

### Meeting Detail Pages

Each meeting row contains a link to the meeting details page:

**URL Pattern:**

```
https://portal.3gpp.org/Home.aspx#/meeting?MtgId=<meeting_id>
```

Where `<meeting_id>` is the unique integer identifier for that meeting.

## Meeting Table Structure

Each meeting list page contains an HTML table with the following columns:

### Columns

1. **Meeting** - Short name (link to meeting details)
2. **Start Date** - Meeting start date
3. **End Date** - Meeting end date (if multi-day)
4. **Location** - Physical location
5. **Status** - Meeting status (e.g., "Upcoming", "Held")
6. **Files** - Link to TDoc FTP directory (CRITICAL for TDoc crawling)
7. **Registration** - Registration deadline

### Files Column - Critical for TDoc Crawling

- **If "Files" column is empty**: Meeting is not yet setup, no TDocs available. Skip for TDoc crawling.
- **If "Files" column has link**: Meeting has TDoc FTP directory set up. Can crawl TDocs for this meeting.

### Meeting Status Values

- **Upcoming** - Future meeting
- **Held** - Past meeting that took place
- **Cancelled** - Meeting was cancelled

### Date Format

Meeting dates in the portal tables are in various formats:

- `15-19 Jan 2024` - Day range
- `15 Jan 2024` - Single day
- `15 January 2024` - Full month name
- `15 January - 19 January 2024` - Separate day and month

**Note:** Always normalize to ISO format (`YYYY-MM-DD`) when storing in database.

## Registration and Check-in

### Registration Process

1. **Registration Deadline**: Usually 2-3 weeks before meeting
2. **Check-in**: On-site registration at venue
3. **Meeting Documents Templates**: Available from 3GPP portal

**Resources:**

- Registration page: <https://portal.3gpp.org/>
- Document templates: <https://www.3gpp.org/delegates-corner/meetings/meeting-document-templates>

## Hosting a Meeting

For companies hosting 3GPP meetings, there are specific requirements and procedures documented by 3GPP.

### Meeting Calendar

The 3GPP Meeting Calendar is available on the 3GU Portal:

- Portal: <https://portal.3gpp.org/#/55930-meetings>
- Shows all TSG meetings
- Includes registration and check-in information

## Cross-References

- **@3gpp-working-groups** - For understanding WG codes in meeting URLs
- **@3gpp-tdocs** - For accessing TDocs from meetings with files links
- **@3gpp-portal-authentication** - For accessing meeting details pages that require login

## Common Mistakes

1. **Using meeting name for database lookups** - Always use `meeting_id` integer
2. **Parsing date strings directly** - Normalize to ISO format first
3. **Skipping empty Files column** - Don't attempt crawl, skip meeting
4. **Year format changes** - Early years use different conventions

## Best Practices

### When Working with Meeting Data

1. **Use meeting_id Integer**: Primary key for database operations
2. **Do NOT parse meeting names**: Use for display only
3. **Check Files column**: Before attempting TDoc crawl
4. **Normalize dates**: Store as ISO format in database
5. **Handle empty Files column**: Skip meeting if no TDocs available

### Common Pitfalls

1. **Parsing meeting names**: Different formats (e.g., `SA4#134` vs `SA4-e`) cannot be reliably parsed
2. **Date format inconsistency**: Multiple formats in portal tables
3. **Empty Files column**: Don't attempt TDoc crawl for meetings without files
4. **Year format changes**: Early years use different conventions

## Resources

- **3GPP Official:** <https://www.3gpp.org/>
- **3GPP Portal:** <https://portal.3gpp.org/>
- **3GPP Work Plan:** <https://www.3gpp.org/specifications-technologies/3gpp-work-plan/>
+304 −0

File added.

Preview size limit exceeded, changes collapsed.

Loading