...
| Code Block |
|---|
fhir-server-url: http://localhost/fhir (Hearth)# Hearth | http://localhost/hapifhir/baseDstu3 # (HAPI FHIR) scenario-iterations: 1000 # how many times a scenario/test will be executed scenario-concurrency: 1 # how many users will execute the given scenario/test identifier_value: # this value is generated for each location resource so searching by identifier always returns a single resource location_id: # set by a FHIR server when a resource is created |
The sample column presents the number of iterations/resources. The memory limit presents if memory limit has been set for FHIR servers and how big it was. The mem column presents how much memory was used during the test. The avg column presents the average time of waiting for a response from all request. The p90 column presents times of 90% of all results (in this case 900 requests). Result times in columns are in milliseconds.
...
| Code Block |
|---|
URL: ${fhir-server-url}/Location
Method: POST
Headers:
Content-Type: application/json
Body:
{
resourceType: "Location",
alias: [ "N007" ],
identifier: {
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/programs/418bdc1d-c303-4bd0-b2d3-d8901150a983" },
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityTypes/ac1d268b-ce10-455f-bf87-9c667da8f060" },
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityOperators/9456c3e9-c4a6-4a28-9e08-47ceb16a4121" },
{ system: "https://test.openlmis.org", value: ${identifier_value} }
},
name: "Lurio, Cuamba",
position: {
longitude: 36.852509
latitude: -14.78975
},
physicalType: {
coding: [
{ system: "http://hl7.org/fhir/location-physical-type", code: "si", display: "Site" }
]
},
partOf: {
reference: "https://test.openlmis.org/api/Location/9b8cfb5a-217a-4261-a64f-16ca06ae79fa"
},
status: "active"
} |
Response times
| FHIR server | Sample | Memory limit | 1st attempt | 2nd attempt | 3rd attempt | 4th attempt | 5th attempt | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | |||
| Hearth | 1 000 | no limit | - | 3 | 4 | - | 3 | 4 | - | 3 | 5 | - | 3 | 5 | - | 3 | 4 |
| HAPI FHIR | - | 105 | 172 | - | 107 | 176 | - | 106 | 174 | - | 106 | 176 | - | 105 | 172 | ||
| Hearth | 10 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 10 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
Update the location
Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource update, the FHIR server will contain a single resource.
...
| Code Block |
|---|
URL: ${fhir-server-url}/Location
Method: PUT
Headers:
Content-Type: application/json
Body:
{
resourceType: "Location",
alias: [ "N007" ],
identifier: {
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/programs/418bdc1d-c303-4bd0-b2d3-d8901150a983" },
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityTypes/ac1d268b-ce10-455f-bf87-9c667da8f060" },
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityOperators/9456c3e9-c4a6-4a28-9e08-47ceb16a4121" },
{ system: "https://test.openlmis.org", value: ${identifier_value} }
},
name: "Lurio, Cuamba (Updated)",
position: {
longitude: 36.852509
latitude: -14.78975
},
physicalType: {
coding: [
{ system: "http://hl7.org/fhir/location-physical-type", code: "si", display: "Site" }
]
},
partOf: {
reference: "https://test.openlmis.org/api/Location/9b8cfb5a-217a-4261-a64f-16ca06ae79fa"
},
status: "active"
}
|
Response times
| FHIR server | Sample | Memory limit | 1st attempt | 2nd attempt | 3rd attempt | 4th attempt | 5th attempt | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | |||
| Hearth | 1 000 | no limit | - | 5 | 6 | - | 5 | 7 | - | 5 | 7 | - | 5 | 7 | - | 4 | 5 |
| HAPI FHIR | - | 115 | 183 | - | 119 | 187 | - | 114 | 182 | - | 114 | 183 | - | 115 | 184 | ||
| Hearth | 10 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 10 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
Get a location by an id
Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource search, the FHIR server will contain only one resource.
...
| Code Block |
|---|
URL: ${fhir-server-url}/Location/${location_id}
Method: GET
Headers:
Accept: application/json |
Response times
| FHIR server | Sample | Memory limit | 1st attempt | 2nd attempt | 3rd attempt | 4th attempt | 5th attempt | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | |||
| Hearth | 1 000 | no limit | - | 3 | 4 | - | 3 | 4 | - | 3 | 4 | - | 3 | 4 | - | 2 | 3 |
| HAPI FHIR | - | 5 | 9 | - | 6 | 10 | - | 5 | 9 | - | 5 | 9 | - | 5 | 9 | ||
| Hearth | 10 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 10 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
Search single location by an identifier
...
| Code Block |
|---|
URL: ${fhir-server-url}/Location?identifier=https://test.openlmis.org|${identifier_value}
Method: GET |
Response times
| FHIR server | Sample | Memory limit | 1st attempt | 2th attempt | 3th attempt | 4th attempt | 5th attempt | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | |||
| Hearth | 1 000 | no limit | - | 4 | 6 | - | 4 | 6 | - | 5 | 7 | - | 4 | 6 | - | 4 | 5 |
| HAPI FHIR | - | 16 | 24 | - | 17 | 26 | - | 17 | 25 | - | 17 | 25 | - | 17 | 25 | ||
| Hearth | 10 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 10 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
Search locations
Note: When an X test is being executed, there are 1000 resources in the FHIR server. The search endpoint returns only 10 resources.
...
| Code Block |
|---|
URL: ${fhir-server-url}/Location
Method: GET |
Response times
| FHIR server | Sample | Memory limit | 1st attempt | 2th attempt | 3th attempt | 4th attempt | 5th attempt | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | mem | avg | p90 | |||
| Hearth | 1 000 | no limit | - | 3 | 4 | - | 3 | 5 | - | 3 | 4 | - | 3 | 5 | - | 3 | 4 |
| HAPI FHIR | - | 19 | 25 | - | 17 | 20 | - | 20 | 28 | - | 20 | 27 | - | 22 | 29 | ||
| Hearth | 10 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 10 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | no limit | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
| Hearth | 20 000 | 1024MB | |||||||||||||||
| HAPI FHIR | |||||||||||||||||
Conclusions
In all attempts, the Hearth FHIR server has better performance than the HAPI FHIR server. Creation and update actions take a lot of more time in the HAPI FHIR than in the Hearth. The search actions have better results but still, Hearth server is faster about 4 times. I noticed that the HAPI FHIR server caches results (not know for how long) for search requests:
...