StockManagement screens issues and improvements [WIP]

This document is intended to list known problems with the StockManagement screens and their mitigation ideas. This should be considered a draft and will be updated by the team working on the Angola implementation, together with the core team to built the correct and desired solution.

The issues have high level issues created:

OLMIS-5404 - Getting issue details... STATUS

OLMIS-5405 - Getting issue details... STATUS

OLMIS-5406 - Getting issue details... STATUS

OLMIS-5407 - Getting issue details... STATUS

OLMIS-5408 - Getting issue details... STATUS

but as a part of this design work, we should come up with more concrete ideas for improvements and then create tickets for specific work that will tackle the performance issues mentioned above.


Known problemDescriptionImprovements
Entering the Physical Inventory screen fetches data for all of the programs

Just attempting to enter the Physical Inventory screen starts the fetch for all stock cards and a lot of other data for ALL of the available programs. This seems to be a waste of time and resources, if I'm only interested in one program.

Details (measured in browser console):

Get StockCardSummaries - 46.5s
Get PhysicalInventories - 0.8s
Get all OrderableFulfills - 14.5s (unnecessary?)
Get all Orderables - 9s
Get all lots - 0.4s


Executing StockCardSummariesv2 endpoint takes about 50 seconds for 20k FTAPs

Current performance of the StockCardSummariesV2 endpoint:

For perftest:
Profiler [GET_STOCK_CARDS_V2]
#012|-- elapsed time [VALIDATE_PARAMS] average 0.000 seconds.
#012|-- elapsed time [GET_STOCK_CARD_SUMMARIES] average 32.969 seconds.
#012|-- elapsed time [TO_DTO] average 0.015 seconds.
#012|-- elapsed time [GET_PAGE] average 0.000 seconds.
#012|-- Total [GET_STOCK_CARDS_V2] average 32.985 seconds.

Profiler [FIND_STOCK_CARD_SUMMARIES_FOR_PARAMS]
#012|-- elapsed time [VALIDATE_VIEW_RIGHTS] average 0.065 seconds.
#012|-- elapsed time [GET_APPROVED_PRODUCTS] average 8.662 seconds.
#012|-- elapsed time [FIND_ORDERABLE_FULFILL_BY_ID] average 22.336 seconds.
#012|-- elapsed time [FIND_STOCK_CARD_BY_PROGRAM_AND_FACILITY] average 0.014 seconds.
#012|-- Total [FIND_STOCK_CARD_SUMMARIES_FOR_PARAMS] average 32.969 seconds.

For uat:
Profiler [GET_STOCK_CARDS_V2]
#012|-- elapsed time [VALIDATE_PARAMS] average 0.000 seconds.
#012|-- elapsed time [GET_STOCK_CARD_SUMMARIES] average 32.105 seconds.
#012|-- elapsed time [TO_DTO] average 0.024 seconds.
#012|-- elapsed time [GET_PAGE] average 0.000 seconds.
#012|-- Total [GET_STOCK_CARDS_V2] average 32.129 seconds.

Profiler [FIND_STOCK_CARD_SUMMARIES_FOR_PARAMS]
#012|-- elapsed time [VALIDATE_VIEW_RIGHTS] average 0.109 seconds.
#012|-- elapsed time [GET_APPROVED_PRODUCTS] average 8.834 seconds.
#012|-- elapsed time [FIND_ORDERABLE_FULFILL_BY_ID] average 23.139 seconds.
#012|-- elapsed time [FIND_STOCK_CARD_BY_PROGRAM_AND_FACILITY] average 0.022 seconds.
#012|-- Total [FIND_STOCK_CARD_SUMMARIES_FOR_PARAMS] average 32.104 seconds

The full result can be found in result.csv (all times are given in seconds).


Additional info:

Number of records on the perftest server:
2,700 stock cards
500,000 stock card line item
500,000 stock event
500,000 stock event line items
20,000 facility type approved products

Number of records on the uat server:
100 stock cards
5,000 stock card line item
10 stock event
100 stock event line items
20,000 facility type approved products

OLMIS-6474 - Getting issue details... STATUS

  • #012|-- elapsed time [FIND_ORDERABLE_FULFILL_BY_ID] average 22.336 seconds improved to 17.734 seconds
All the orderables for matching FTAPs are queried by id, just so they can be selected by name on product addition modalWe can look into a different strategy for retrieving orderable names - caching/limited representation/only allow retrieving them on search
Can only query ~40 orderables by UUID at a time with GET requestThe UI is querying for hundreds/thousands of orderables by IDs, and one request can only fit ~40 UUIDs. This means the UI is firing many requests to retrieve all of the data. If this is absolutely necessary to retrieve them, maybe other params could be used.
Unnecessary call to /orderableFulfills

When entering the physical inventory screen, the UI sends a request to both StockCardSummaries and to OrderableFulfills. All the info retrieved by OrderableFulfills is already contained within StockCardSummaries (this endpoint also uses the Orderable Fulfills to build its response) and so, it shouldn't be required to query for it again.

There's a chance that there's something missing in StockCardSummariesV2.


StockCards must recalculate current SoH

Stock on Hand for the given StockCard is not stored anywhere. Each time the user is interested in the SoH (calling the summaries endpoint), the SoH must be recalculated, which is an expensive operation (it involves retrieving all the line items - aka stock movement, ordering them by date of occurence, and then going one by one to calculate the current stock on hand. The more stock movements, the more to calculate.

  1. We don't want SoH in line items
  2. We wouldn't want to update StockCardLineItems and StockEvents

The options:

  1. Store SoH in StockCards (may not be complete)
  2. Event snapshot with SoH as of any given date (including most recent one) - worth checking on event sourcing and snapshotting

The biggest part of the request take GET_APPROVED_PRODUCTS and FIND_ORDERABLE_FULFILL_BY_ID stepsBoth steps relate to calls toward reference data. First of them gets FTAPs, the seconde orderableFullfils. During getting orderableFullfils multiple calls are made. It is because of the need to split too long URL.

Next steps:

ProblemSolutionsTicket
StockCardSummariesV2 is slowStart with improving the profiling for the endpoint - currently the profiler is added on the controller layer, but over 99% of the time the enpoint spends on the service/repository layer.
AO-137 - Getting issue details... STATUS
The biggest part of the request take GET_APPROVED_PRODUCTS and FIND_ORDERABLE_FULFILL_BY_ID steps
  • allow searching orderableFulfills by facilityId and programId

  • get rid of call for FTAPs - most probably the data from orderableFulfills is sufficient

OLMIS-6614 - Getting issue details... STATUS


OpenLMIS: the global initiative for powerful LMIS software