97a6566e3d
* Fix API stalls: non-blocking switch worker, threaded searches, indexed Videos page Three response-time fixes found by benchmarking the read-only API endpoints hot and cold on real libraries: * `switch_worker` runs on the Sanic worker's event loop every 0.1s and called the blocking `multiprocessing.Event.wait(timeout=1)`. With no switch pending (the normal state) this froze the loop for a second at a time; about a third of all requests, including trivial ones, took ~1,000 ms. It now polls `is_set()` and returns immediately. * The search handlers (videos, files, archives, docs, zims) and `/api/statistics` ran synchronous SQLite queries on the event loop, so one slow search stalled every other request on that worker. They now run via `asyncio.to_thread`. * The unfiltered Videos page (`POST /api/videos/search` with only an order) used `mimetype LIKE 'video/%' OR ...` plus `COUNT(*) OVER()`, which visits every video's index entry before LIMIT applies: seconds cold on a large library on a spinning disk, and ~10 s hot for a deep page. It now seeks each exact mimetype on the (mimetype, effective_datetime) index for `limit + offset` rows and merges them, with the total from a separate count cached for 60 s. Results are identical to the generic query. Also adds `scripts/api_perf.py` (times every read-only endpoint against a host, optionally restarting the API over SSH for cold numbers) and `scripts/api_perf_compare.py` (merges per-host results). Host profiles live in a gitignored JSON file; an example is included. Measured on an 8-core docker host with a 10 GB library: median hot request 1,007 ms -> 22 ms; Videos page 1 hot 287 -> 108 ms, page 50 1,345 -> 120 ms. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Serve the video description from its own endpoint instead of Video.__json__ `Video.__json__` computed `description` as `c_text or get_video_description()`, where the fallback opens and parses the video's info json from disk. The list endpoints serialize every result with `__json__`, so any page containing videos without an indexed description (common: many videos have none) read their info json files on every request, cold, for a field the list never displays. The description is not part of search either: search runs against the FTS copy of `c_text`, so the fallback never fed the index. Drop `description` from `__json__` and serve it from `GET /api/videos/<id>/description`, like comments and captions. The endpoint returns the indexed `c_text`, falling back to the info json only there, for a row indexed before descriptions were stored. The player page fetches it through `useVideoExtras` alongside the comments. Tests assert the list and detail responses never touch the info json, and cover the endpoint's indexed, fallback, and not-found cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Address review: thread-local zim sessions, fast-path equivalence test, comments * Zim search, single-zim search, and search estimates run libzim off the event loop, each opening its own session inside the worker thread rather than passing the request's SQLAlchemy Session across threads. * The Videos fast-path test now runs the same requests through the generic query and asserts identical rows and paging for both date orders. The one intentional difference is documented: past the end, the fast path still reports the true total. * switch_worker returns before its try block so the empty path does not clear the event; its comment states the invariant only. The copied "synchronous SQLite" comments are reduced to one line in one place. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| cypress | ||
| public | ||
| scripts | ||
| src | ||
| .gitignore | ||
| cypress.config.js | ||
| package-lock.json | ||
| package.json | ||
| README.md | ||
| serve.json | ||
| tsconfig.json | ||
Getting Started with Create React App
This project was bootstrapped with Create React App.
Available Scripts
In the project directory, you can run:
npm start
Runs the app in the development mode.
Open http://localhost:3000 to view it in your browser.
The page will reload when you make changes.
You may also see any lint errors in the console.
npm test
Launches the test runner in the interactive watch mode.
See the section about running tests for more
information.
npm run build
Builds the app for production to the build folder.
It correctly bundles React in production mode and optimizes the build for the best performance.
The build is minified and the filenames include the hashes.
Your app is ready to be deployed!
See the section about deployment for more information.
npm run eject
Note: this is a one-way operation. Once you eject, you can't go back!
If you aren't satisfied with the build tool and configuration choices, you can eject at any time. This command will
remove the single build dependency from your project.
Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right
into your project so you have full control over them. All of the commands except eject will still work, but they will
point to the copied scripts so you can tweak them. At this point you're on your own.
You don't have to ever use eject. The curated feature set is suitable for small and middle deployments, and you
shouldn't feel obligated to use this feature. However we understand that this tool wouldn't be useful if you couldn't
customize it when you are ready for it.
Learn More
You can learn more in the Create React App documentation.
To learn React, check out the React documentation.
Code Splitting
This section has moved here: https://facebook.github.io/create-react-app/docs/code-splitting
Analyzing the Bundle Size
This section has moved here: https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size
Making a Progressive Web App
This section has moved here: https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app
Advanced Configuration
This section has moved here: https://facebook.github.io/create-react-app/docs/advanced-configuration
Deployment
This section has moved here: https://facebook.github.io/create-react-app/docs/deployment
npm run build fails to minify
This section has moved here: https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify