summary

Order created without releasing requisition

Description

During Malawi's support, we found that sometimes the order is created but requisition still has APPROVED status. We haven’t managed to reproduce it on the core servers and we need to be able to do so to verify it.

Acceptance Criteria:

The requisition should always be released before creating an order

Approach:

This might be caused by users attempting to convert the same requisitions to order multiple times - it may cause one of the requests to rollback the data. We can try to repro by doing the same - sending multiple converts to order for the same requisitions
At the same time, we can look to any (quick) performance improvements - if we convert the requisitions faster, there’s smaller change someone attempts to release the same requisitions again
We can also reach to Malawi and ask them to use their test servers and to provide any more details that will help us to reproduce
As a last resort, we can try to manually throw an error somewhere in the convert to order method to see what happens

Possible fixes:

If we discover the order already exists for the requisition, and the requisition is in the approved state, we just change its status to RELEASED
We can send a PUT request instead of the POST request to the fulfillment service - if the order already exists it should do nothing/update and move on.
Use idempotency key with convert to order request - we could store IDs of the requisitions that are currently being processed. If there’s another request that is started and at least one requisition is on the list, we don’t accept the request

Error when the order already exists:

Environment

None

Activity

Show:

Sebastian Brudziński

3 days ago

Profiler [SAVE_SHIPMENT]
[SAVE_SHIPMENT_TO_DB] 150.799 seconds.
[CREATE_POD] 0.001 seconds.
[SAVE_POD_TO_DB] 15.247 seconds.
Total [SAVE_SHIPMENT] 166.047 seconds.

It looks like we should take a look at the shipment save to the database - this is taking the majority of the time.

Sebastian Brudziński

March 31, 2021, 9:23 AM

After some investigations, we are planning to focus on two things to fix the problem

Performance of POST /orders/batch (this is the majority of the time when converting an order)
Adjustments of the behavior of POST /orders/batch - it will not attempt to create an order anymore if the endpoint finds that the order with provided externalId already exists. In this scenario, it will still return 200 OK even if some orders already existed

Sebastian Brudziński

March 26, 2021, 3:31 PM

Edited

Info from MW

1. Do you know if it is possible that users refresh the page when converting takes a long time and try to convert again without waiting for the first response from the server?
Yes after waiting 10 minutes the user refreshed the page to try again when he first experienced the issue

2. Do you know which users generally get this error and for which facilities?
Supplying depot was CMST North, facilities where from Rhumpi and Nkhotakota district. User can't remember the specific facilities.

3. Does it occur no matter how many forms the user is converting?
The user tried converting the entire page that is when they reported the issue that orders were not being removed After you ran the script to remove the scripts the user reported trying to convert 2-3 and had success When the user tries to convert more than 3 they experience the issue of the forms taking forever to convert and not removed from the convert to order list.

Klaudia Pałkowska

March 26, 2021, 1:04 PM

NOTE: Malawi’s users who usually experience this issue are kngoma and efoster.

Klaudia Pałkowska

March 26, 2021, 7:45 AM

Not at all. As far as I remember, I wasn’t able to reproduce it on Malawi’s UAT in the past but we’ll see. Maybe someone will find proper reproduction steps and reproduce the issue.