Commit f2c567d3 authored by Jan Reimes's avatar Jan Reimes
Browse files

Fix file_size=0 preventing size resolution in TDoc queries 

_prepare_tdoc normalized file_size=None -> 0 for bulk insert consistency,
but resolve_tdoc_file_sizes skipped 0 (treated as truthy/known). This
made every record show '?' in the Size (KB) column.

- Remove file_size normalization from _prepare_tdoc (None means unknown)
- Change resolve guard to truthiness check (catches legacy 0 values)
- Backfilled 21232 records in DB: file_size=0 -> NULL
parent 7deb92da

src/tdoc_crawler/database/tdocs.py

+0 −2
Original line number Diff line number Diff line
@@ -343,8 +343,6 @@ class TDocDatabase(MeetingDatabase): 
            updates["date_created"] = metadata.date_retrieved or utc_now()

        if metadata.tbid is None:

            updates["tbid"] = 0

        if metadata.file_size is None:

            updates["file_size"] = 0

        if updates:

            return TDocDatabase._clone_tdoc(metadata, updates)

        return metadata

src/tdoc_crawler/tdocs/operations/checkout.py

+1 −1
Original line number Diff line number Diff line
@@ -45,7 +45,7 @@ def resolve_tdoc_file_sizes(results: list[TDocMetadata], checkout_dir: Path) -> 
    Updates ``record.file_size`` in-place.

    """

    for record in results:

        if record.file_size is not None or not record.url:

        if record.file_size or not record.url:

            continue

        tdoc_dir = get_checkout_path(record, checkout_dir)

        if not tdoc_dir.exists():