Context
Raised during review of #495 (#495 (comment)):
what is the use case of getting all functions? I hope this is not in the function viewer? That one should only fetch the function that are shown on the page, not to overload the API.
Confirmed: this is the function viewer, and it does fetch everything.
What happens today
FunctionList.tsx calls listFunctions() unconditionally on mount, which hits GET /flask/osparc/list_functions.
- The backend route (
flaskapi/src/mmux_flaskapi/blueprints/osparc.py) resolves this via _get_all_items() (flaskapi/src/mmux_flaskapi/utils/helpers.py), which loops through oSPARC's paginated API (50 items/page) until every function is retrieved — no limit tied to what's actually rendered.
- On top of that,
FunctionList.tsx's fetchFunctions() then fans out one additional API call per function (getFunctionJobCollections(), via Promise.all) to compute campaign/job counts for each row.
- The MUI
DataGrid only paginates client-side, after everything has already been fetched — so it does nothing to reduce API load.
Net effect: as the number of registered oSPARC functions grows, opening the function viewer means O(N) backend pagination calls to oSPARC plus O(N) more calls for job counts, all on a single page load.
Proposed fix (needs design decision)
One of:
- Add real pagination to
/flask/osparc/list_functions (limit/offset query params) and switch DataGrid to server-side pagination, fetching only the page currently shown.
- Add a lighter list/summary endpoint that avoids the N+1 job-collection-count fan-out (e.g. batch job counts, or fetch them lazily/on-demand per row instead of upfront for all rows).
Files involved
node/src/components/setup/FunctionList.tsx
node/src/utils/functionUtils.ts
flaskapi/src/mmux_flaskapi/blueprints/osparc.py
flaskapi/src/mmux_flaskapi/utils/helpers.py (_get_all_items)
Context
Raised during review of #495 (#495 (comment)):
Confirmed: this is the function viewer, and it does fetch everything.
What happens today
FunctionList.tsxcallslistFunctions()unconditionally on mount, which hitsGET /flask/osparc/list_functions.flaskapi/src/mmux_flaskapi/blueprints/osparc.py) resolves this via_get_all_items()(flaskapi/src/mmux_flaskapi/utils/helpers.py), which loops through oSPARC's paginated API (50 items/page) until every function is retrieved — no limit tied to what's actually rendered.FunctionList.tsx'sfetchFunctions()then fans out one additional API call per function (getFunctionJobCollections(), viaPromise.all) to compute campaign/job counts for each row.DataGridonly paginates client-side, after everything has already been fetched — so it does nothing to reduce API load.Net effect: as the number of registered oSPARC functions grows, opening the function viewer means O(N) backend pagination calls to oSPARC plus O(N) more calls for job counts, all on a single page load.
Proposed fix (needs design decision)
One of:
/flask/osparc/list_functions(limit/offsetquery params) and switchDataGridto server-side pagination, fetching only the page currently shown.Files involved
node/src/components/setup/FunctionList.tsxnode/src/utils/functionUtils.tsflaskapi/src/mmux_flaskapi/blueprints/osparc.pyflaskapi/src/mmux_flaskapi/utils/helpers.py(_get_all_items)