Order created without releasing requisition
Description
During Malawi's support, we found that sometimes the order is created but requisition still has APPROVED status. We haven’t managed to reproduce it on the core servers and we need to be able to do so to verify it.
Acceptance Criteria:
The requisition should always be released before creating an order
Approach:
This might be caused by users attempting to convert the same requisitions to order multiple times - it may cause one of the requests to rollback the data. We can try to repro by doing the same - sending multiple converts to order for the same requisitions
At the same time, we can look to any (quick) performance improvements - if we convert the requisitions faster, there’s smaller change someone attempts to release the same requisitions again
We can also reach to Malawi and ask them to use their test servers and to provide any more details that will help us to reproduce
As a last resort, we can try to manually throw an error somewhere in the convert to order method to see what happens
Possible fixes:
If we discover the order already exists for the requisition, and the requisition is in the approved state, we just change its status to RELEASED
We can send a PUT request instead of the POST request to the fulfillment service - if the order already exists it should do nothing/update and move on.
Use idempotency key with convert to order request - we could store IDs of the requisitions that are currently being processed. If there’s another request that is started and at least one requisition is on the list, we don’t accept the request
Error when the order already exists:
Environment
Activity
Profiler [SAVE_SHIPMENT]
[SAVE_SHIPMENT_TO_DB] 150.799 seconds.
[CREATE_POD] 0.001 seconds.
[SAVE_POD_TO_DB] 15.247 seconds.
Total [SAVE_SHIPMENT] 166.047 seconds.
It looks like we should take a look at the shipment save to the database - this is taking the majority of the time.
After some investigations, we are planning to focus on two things to fix the problem
Performance of POST /orders/batch (this is the majority of the time when converting an order)
Adjustments of the behavior of POST /orders/batch - it will not attempt to create an order anymore if the endpoint finds that the order with provided externalId already exists. In this scenario, it will still return 200 OK even if some orders already existed
Info from MW
1. Do you know if it is possible that users refresh the page when converting takes a long time and try to convert again without waiting for the first response from the server?
Yes after waiting 10 minutes the user refreshed the page to try again when he first experienced the issue
2. Do you know which users generally get this error and for which facilities?
Supplying depot was CMST North, facilities where from Rhumpi and Nkhotakota district. User can't remember the specific facilities.
3. Does it occur no matter how many forms the user is converting?
The user tried converting the entire page that is when they reported the issue that orders were not being removed
After you ran the script to remove the scripts the user reported trying to convert 2-3 and had success
When the user tries to convert more than 3 they experience the issue of the forms taking forever to convert and not removed from the convert to order list.
NOTE: Malawi’s users who usually experience this issue are kngoma and efoster.
Not at all. As far as I remember, I wasn’t able to reproduce it on Malawi’s UAT in the past but we’ll see. Maybe someone will find proper reproduction steps and reproduce the issue.