wrolpi/modules/archive/api.py
Roland 58de8c7c5b
Upgrading Sanic and yt-dlp. Major changes occurred because Sanic changed how multiprocessing works.
Python 3.11 is now required, old versions were causing unpredictability in tests.  (Sanic does not yet
support 3.12)

Sanic has been upgraded to 23.6.0, which is the latest version that avoids this bug:
 https://github.com/sanic-org/sanic/issues/2921

New strategy for multiprocessing is to create all multiprocessing tools in one process, then fork
to other processes.  The previous strategy was to declare multiprocessing tools at the top of every
file, or wherever they were needed at import/creation.  Now all multiprocessing tools are attached to the
app.shared_ctx.  This means `api_app` is imported in many, many places.

This forced a change in how the DownloadManager works.  Previously, it would continually run
download workers which would pull downloads from a multiprocessing.Queue.  Now, a single worker checks
for new downloads and sends a Sanic signal.

Flags have been reworked to use the `api_app`.  I removed the `which` flag functionality because the
`which` are called at import and needed their own multiprocessing.Event.
2024-04-30 10:55:25 -06:00

72 lines
2.6 KiB
Python

from http import HTTPStatus
from sanic import response, Request, Blueprint
from sanic_ext import validate
from sanic_ext.extensions.openapi import openapi
from wrolpi.common import logger, wrol_mode_check, api_param_limiter
from wrolpi.errors import ValidationError
from wrolpi.api_utils import json_response
from wrolpi.schema import JSONErrorResponse
from . import lib, schema
NAME = 'archive'
archive_bp = Blueprint('Archive', '/api/archive')
logger = logger.getChild(__name__)
@archive_bp.get('/<archive_id:int>')
@openapi.description('Get an archive')
@openapi.response(HTTPStatus.NOT_FOUND, JSONErrorResponse)
async def get_archive(_: Request, archive_id: int):
archive = lib.get_archive(archive_id=archive_id)
archive_file_group = archive.file_group.__json__()
history = [i.file_group.__json__() for i in archive.history]
return json_response({'file_group': archive_file_group, 'history': history})
@archive_bp.delete('/<archive_ids:int>', name='archive_delete_one')
@archive_bp.delete('/<archive_ids:[0-9,]+>', name='archive_delete_many')
@openapi.description('Delete an individual archive')
@openapi.response(HTTPStatus.NOT_FOUND, JSONErrorResponse)
@wrol_mode_check
async def delete_archive(_: Request, archive_ids: str):
try:
archive_ids = [int(i) for i in str(archive_ids).split(',')]
except ValueError:
raise ValidationError('Could not parse archive ids')
lib.delete_archives(*archive_ids)
return response.empty()
@archive_bp.get('/domains')
@openapi.summary('Get a list of all Domains and their Archive statistics')
@openapi.response(200, schema.GetDomainsResponse, "The list of domains")
async def get_domains(_: Request):
domains = lib.get_domains()
return json_response({'domains': domains, 'totals': {'domains': len(domains)}})
archive_limit_limiter = api_param_limiter(100)
@archive_bp.post('/search')
@openapi.definition(
summary='A File search with more filtering related to Archives',
body=schema.ArchiveSearchRequest,
)
@openapi.response(HTTPStatus.OK, schema.ArchiveSearchResponse)
@openapi.response(HTTPStatus.NOT_FOUND, JSONErrorResponse)
@validate(schema.ArchiveSearchRequest)
async def search_archives(_: Request, body: schema.ArchiveSearchRequest):
search_str = body.search_str
domain = body.domain
limit = archive_limit_limiter(body.limit)
offset = body.offset or 0
file_groups, total = lib.search_archives(search_str, domain, limit, offset, body.order_by, body.tag_names,
body.headline)
ret = dict(file_groups=file_groups, totals=dict(file_groups=total))
return json_response(ret)