Overview
The following document contains information about FHIR servers performance: Hearth and HAPI FHIR. Tests contain create, update and read actions on a location resource. Each section presents the request structure and response times. To execute tests, the Taurus tool has been used. The following machine has been used for tests:
Memory: 7.7 GiB Processor: Intel Core i7-6700HQ 2.60GHz x 8 Graphics: GeForce GTX 960M/Pcie/SSE2 OS: Ubuntu 18.04.1 LTS (64-bit)
The following variables have been set and used to avoid duplication in tests. Some of them are used in request bodies.
fhir-server-url: http://localhost/fhir (Hearth) | http://localhost/hapifhir/baseDstu3 (HAPI FHIR) scenario-iterations: 1000 # how many times a scenario/test will be executed scenario-concurrency: 1 # how many users will execute the given scenario/test identifier_value: # this value is generated for each location resource so searching by identifier always returns a single resource location_id: # set by a FHIR server when a resource is created
The avg column presents the average time of waiting for a response from all request. The p90 column presents times of 90% of all results (in this case 900 requests). Result times in columns are in milliseconds.
Create a new location
Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource creation, the FHIR server will not contain any resources.
Request body
URL: ${fhir-server-url}/Location
Method: POST
Headers:
Content-Type: application/json
Body:
{
resourceType: "Location",
alias: [ "N007" ],
identifier: {
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/programs/418bdc1d-c303-4bd0-b2d3-d8901150a983" },
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityTypes/ac1d268b-ce10-455f-bf87-9c667da8f060" },
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityOperators/9456c3e9-c4a6-4a28-9e08-47ceb16a4121" },
{ system: "https://test.openlmis.org", value: ${identifier_value} }
},
name: "Lurio, Cuamba",
position: {
longitude: 36.852509
latitude: -14.78975
},
physicalType: {
coding: [
{ system: "http://hl7.org/fhir/location-physical-type", code: "si", display: "Site" }
]
},
partOf: {
reference: "https://test.openlmis.org/api/Location/9b8cfb5a-217a-4261-a64f-16ca06ae79fa"
},
status: "active"
}
Response times
| FHIR server | 1st attempt | 2nd attempt | 3rd attempt | 4th attempt | 5th attempt | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| avg | p90 | avg | p90 | avg | p90 | avg | p90 | avg | p90 | |
| Hearth | 3 | 4 | 3 | 4 | 3 | 5 | 3 | 5 | 3 | 4 |
| HAPI FHIR | 105 | 172 | 107 | 176 | 106 | 174 | 106 | 176 | 105 | 172 |
Update the location
Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource update, the FHIR server will contain a single resource.
Request body
URL: ${fhir-server-url}/Location
Method: PUT
Headers:
Content-Type: application/json
Body:
{
resourceType: "Location",
alias: [ "N007" ],
identifier: {
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/programs/418bdc1d-c303-4bd0-b2d3-d8901150a983" },
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityTypes/ac1d268b-ce10-455f-bf87-9c667da8f060" },
{ system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityOperators/9456c3e9-c4a6-4a28-9e08-47ceb16a4121" },
{ system: "https://test.openlmis.org", value: ${identifier_value} }
},
name: "Lurio, Cuamba (Updated)",
position: {
longitude: 36.852509
latitude: -14.78975
},
physicalType: {
coding: [
{ system: "http://hl7.org/fhir/location-physical-type", code: "si", display: "Site" }
]
},
partOf: {
reference: "https://test.openlmis.org/api/Location/9b8cfb5a-217a-4261-a64f-16ca06ae79fa"
},
status: "active"
}
Response times
| FHIR server | 1st attempt | 2nd attempt | 3rd attempt | 4th attempt | 5th attempt | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| avg | p90 | avg | p90 | avg | p90 | avg | p90 | avg | p90 | |
| Hearth | 5 | 6 | 5 | 7 | 5 | 7 | 5 | 7 | 4 | 5 |
| HAPI FHIR | 115 | 183 | 119 | 187 | 114 | 182 | 114 | 183 | 115 | 184 |
Get a location by an id
Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource search, the FHIR server will contain only one resource.
Request body
URL: ${fhir-server-url}/Location/${location_id}
Method: GET
Headers:
Accept: application/json
Response times
| FHIR server | 1st attempt | 2nd attempt | 3rd attempt | 4th attempt | 5th attempt | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| avg | p90 | avg | p90 | avg | p90 | avg | p90 | avg | p90 | |
| Hearth | 3 | 4 | 3 | 4 | 3 | 4 | 3 | 4 | 2 | 3 |
| HAPI FHIR | 5 | 9 | 6 | 10 | 5 | 9 | 5 | 9 | 5 | 9 |
Search single location by an identifier
Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource search, the FHIR server will contain only one resource.
Request body
URL: ${fhir-server-url}/Location?identifier=https://test.openlmis.org|${identifier_value}
Method: GET
Response times
| FHIR server | 1st attempt | 2th attempt | 3th attempt | 4th attempt | 5th attempt | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| avg | p90 | avg | p90 | avg | p90 | avg | p90 | avg | p90 | |
| Hearth | 4 | 6 | 4 | 6 | 5 | 7 | 4 | 6 | 4 | 5 |
| HAPI FHIR | 16 | 24 | 17 | 26 | 17 | 25 | 17 | 25 | 17 | 25 |
Search locations
Note: When an X test is being executed, there are 1000 resources in the FHIR server. The search endpoint returns only 10 resources.
Request body
URL: ${fhir-server-url}/Location
Method: GET
Response times
| FHIR server | 1st attempt | 2th attempt | 3th attempt | 4th attempt | 5th attempt | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| avg | p90 | avg | p90 | avg | p90 | avg | p90 | avg | p90 | |
| Hearth | 3 | 4 | 3 | 5 | 3 | 4 | 3 | 5 | 3 | 4 |
| HAPI FHIR | 19 | 25 | 17 | 20 | 20 | 28 | 20 | 27 | 22 | 29 |
Conclusions
In all attempts, the Hearth FHIR server has better performance than the HAPI FHIR server. Creation and update actions take a lot of more time in the HAPI FHIR than in the Hearth. The search actions have better results but still, Hearth server is faster about 4 times. I noticed that the HAPI FHIR server caches results (not know for how long) for search requests:
hapi_1 | 2018-09-05 13:42:12.190 [http-nio-8080-exec-7] INFO c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:151] Search entity marked as finished hapi_1 | 2018-09-05 13:42:12.217 [http-nio-8080-exec-5] INFO c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:301] Reusing search f36b5bb4-d58c-4836-83fe-448225372c85 from cache hapi_1 | 2018-09-05 13:42:12.228 [http-nio-8080-exec-5] INFO c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:151] Search entity marked as finished hapi_1 | 2018-09-05 13:42:12.245 [http-nio-8080-exec-8] INFO c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:301] Reusing search f36b5bb4-d58c-4836-83fe-448225372c85 from cache hapi_1 | 2018-09-05 13:42:12.250 [http-nio-8080-exec-8] INFO c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:151] Search entity marked as finished hapi_1 | 2018-09-05 13:42:12.265 [http-nio-8080-exec-1] INFO c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:301] Reusing search f36b5bb4-d58c-4836-83fe-448225372c85 from cache hapi_1 | 2018-09-05 13:42:12.269 [http-nio-8080-exec-1] INFO c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:151] Search entity marked as finished hapi_1 | 2018-09-05 13:42:12.288 [http-nio-8080-exec-4] INFO c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:301] Reusing search f36b5bb4-d58c-4836-83fe-448225372c85 from cache
Performance results
The following zip file contains all result data of all attempts