Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 19 Next »

Overview

The following document contains information about FHIR servers performance: Hearth and HAPI FHIR. Tests contain create, update and read actions on a location resource. Each section presents the request structure and response times. To execute tests, the Taurus tool has been used. The following machine has been used for tests:

Memory: 7.7 GiB
Processor: Intel Core i7-6700HQ 2.60GHz x 8
Graphics: GeForce GTX 960M/Pcie/SSE2
OS: Ubuntu 18.04.1 LTS (64-bit)

The following variables have been set and used to avoid duplication in tests. Some of them are used in request bodies.

fhir-server-url: http://localhost/fhir # Hearth
                 http://localhost/hapifhir/baseDstu3  # HAPI FHIR
scenario-iterations: # how many times a scenario/test will be executed
scenario-concurrency: 1 # how many users will execute the given scenario/test
identifier_value: # this value is generated for each location resource so searching by identifier always returns a single resource
location_id: # set by a FHIR server when a resource is created

The sample column presents the number of iterations/resources. The memory limit presents if memory limit has been set for FHIR servers and how big it was. The mem column presents how much memory was used by the docker container before and after all tests - that is why values are the same in all result tables. The avg column presents the average time of waiting for a response from all request. The p90 column presents times of 90% of all results (in this case 900 requests). Result times in columns are in milliseconds

Create a new location

Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource creation, the FHIR server will not contain any resources.

Request body

URL: ${fhir-server-url}/Location
Method: POST
Headers:
  Content-Type: application/json
Body:
  {
    resourceType: "Location",
	alias: [ "N007" ],
	identifier: {
	  { system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/programs/418bdc1d-c303-4bd0-b2d3-d8901150a983" },
	  { system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityTypes/ac1d268b-ce10-455f-bf87-9c667da8f060" },
	  { system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityOperators/9456c3e9-c4a6-4a28-9e08-47ceb16a4121" },
	  { system: "https://test.openlmis.org", value: ${identifier_value} }
	},
	name: "Lurio, Cuamba",
	position: {
	  longitude: 36.852509
	  latitude: -14.78975
	},
	physicalType: {
	  coding: [
	    { system: "http://hl7.org/fhir/location-physical-type", code: "si", display: "Site" }
	  ]
	},
	partOf: {
	  reference: "https://test.openlmis.org/api/Location/9b8cfb5a-217a-4261-a64f-16ca06ae79fa"
	},
	status: "active"
  }

Response times

FHIR serverSampleMemory limit

1st attempt

2nd attempt3rd attempt4th attempt5th attempt
memavgp90memavgp90memavgp90memavgp90memavgp90
Hearth1 000no limit-34-34-35-35-34
HAPI FHIR-105172-107176-106174-106176-105172
Hearth10 000no limit

158MiB/118MiB

24175MiB/165MiB24172MiB/136MiB34------
HAPI FHIR992MiB/1.399GiB103176982MiB/1.442GiB70129957MiB/1.357GiB102172------

Update the location

Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource update, the FHIR server will contain a single resource.

Request body

URL: ${fhir-server-url}/Location
Method: PUT
Headers:
  Content-Type: application/json
Body:
  {
    resourceType: "Location",
	alias: [ "N007" ],
	identifier: {
	  { system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/programs/418bdc1d-c303-4bd0-b2d3-d8901150a983" },
	  { system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityTypes/ac1d268b-ce10-455f-bf87-9c667da8f060" },
	  { system: "urn:ietf:rfc:3986", value: "https://test.openlmis.org/api/facilityOperators/9456c3e9-c4a6-4a28-9e08-47ceb16a4121" },
	  { system: "https://test.openlmis.org", value: ${identifier_value} }
	},
	name: "Lurio, Cuamba (Updated)",
	position: {
	  longitude: 36.852509
	  latitude: -14.78975
	},
	physicalType: {
	  coding: [
	    { system: "http://hl7.org/fhir/location-physical-type", code: "si", display: "Site" }
	  ]
	},
	partOf: {
	  reference: "https://test.openlmis.org/api/Location/9b8cfb5a-217a-4261-a64f-16ca06ae79fa"
	},
	status: "active"
  }

Response times

FHIR serverSampleMemory limit1st attempt2nd attempt3rd attempt4th attempt5th attempt
memavgp90memavgp90memavgp90memavgp90memavgp90
Hearth1 000no limit-56-57-57-57-45
HAPI FHIR-115183-119187-114182-114183-115184
Hearth10 000no limit158MiB/118MiB1219175MiB/165MiB1320172MiB/136MiB1320------
HAPI FHIR992MiB/1.399GiB112179982MiB/1.442GiB76143957MiB/1.357GiB114181------

Get a location by an id

Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource search, the FHIR server will contain only one resource.

Request body

URL: ${fhir-server-url}/Location/${location_id}
Method: GET
Headers:
  Accept: application/json

Response times

FHIR serverSampleMemory limit1st attempt2nd attempt3rd attempt4th attempt5th attempt
memavgp90memavgp90memavgp90memavgp90memavgp90
Hearth1 000no limit-34-34-34-34-23
HAPI FHIR-59-610-59-59-59
Hearth10 000no limit158MiB/118MiB610175MiB/165MiB610172MiB/136MiB710------
HAPI FHIR992MiB/1.399GiB48982MiB/1.442GiB47957MiB/1.357GiB47------

Search single location by an identifier

Note: When an X test is being executed, there are X - 1 resources in the FHIR server. For the first resource search, the FHIR server will contain only one resource.

Request body

URL: ${fhir-server-url}/Location?identifier=https://test.openlmis.org|${identifier_value}
Method: GET

Response times

FHIR serverSampleMemory limit1st attempt2th attempt3th attempt4th attempt5th attempt
memavgp90memavgp90memavgp90memavgp90memavgp90
Hearth1 000no limit-46-46-57-46-45
HAPI FHIR-1624-1726-1725-1725-1725
Hearth10 000no limit158MiB/118MiB1729175MiB/165MiB1729172MiB/136MiB1830------
HAPI FHIR992MiB/1.399GiB1420982MiB/1.442GiB1019957MiB/1.357GiB1319------

Search locations

Note: When an X test is being executed, there are as many resources as the number in the sample column in the FHIR server. The search endpoint returns only 10 resources.

Request body

URL: ${fhir-server-url}/Location
Method: GET

Response times

FHIR serverSampleMemory limit1st attempt2th attempt3th attempt4th attempt5th attempt
memavgp90memavgp90memavgp90memavgp90memavgp90
Hearth1 000no limit-34-35-34-35-34
HAPI FHIR-1925-1720-2028-2027-2229
Hearth10 000no limit158MiB/118MiB34175MiB/165MiB34172MiB/136MiB34------
HAPI FHIR992MiB/1.399GiB1924982MiB/1.442GiB2025957MiB/1.357GiB1721------

Search locations by name

Note: When an X test is being executed, there are as many resources as the number in the sample column in the FHIR server. The search endpoint returns only 10 resources.

Request body

URL: ${fhir-server-url}/Location?name:contains=Lurio
Method: GET

Response times

FHIR serverSampleMemory limit1st attempt

2th attempt

3th attempt

4th attempt

5th attempt

memavgp90memavgp90memavgp90memavgp90memavgp90
Hearth1 000no limit---------------
HAPI FHIR---------------
Hearth10 000no limit158MiB/118MiB1213175MiB/165MiB1214172MiB/136MiB1214------
HAPI FHIR992MiB/1.399GiB1924982MiB/1.442GiB1923957MiB/1.357GiB1924------

Search locations by partOf

Note: When an X test is being executed, there are as many resources as the number in the sample column in the FHIR server. The search endpoint returns only 10 resources.

Request body

URL: ${fhir-server-url}/Location?partof=https://test.openlmis.org/api/Location/9b8cfb5a-217a-4261-a64f-16ca06ae79fa
Method: GET

Response times

FHIR serverSampleMemory limit1st attempt

2th attempt

3th attempt

4th attempt

5th attempt

memavgp90memavgp90memavgp90memavgp90memavgp90
Hearth1 000no limit---------------
HAPI FHIR---------------
Hearth10 000no limit158MiB/118MiB1112175MiB/165MiB1113172MiB/136MiB1113------
HAPI FHIR992MiB/1.399GiB2025982MiB/1.442GiB1923957MiB/1.357GiB1823------

Search locations by status

Note: When an X test is being executed, there are as many resources as the number in the sample column in the FHIR server. The search endpoint returns only 10 resources.

Request body

URL: ${fhir-server-url}/Location?status:exact=active
Method: GET

Response times

FHIR serverSampleMemory limit1st attempt

2th attempt

3th attempt

4th attempt

5th attempt

memavgp90memavgp90memavgp90memavgp90memavgp90
Hearth1 000no limit---------------
HAPI FHIR---------------
Hearth10 000no limit158MiB/118MiB1213175MiB/165MiB1113172MiB/136MiB1214------
HAPI FHIR992MiB/1.399GiB2026982MiB/1.442GiB1823957MiB/1.357GiB1720------

Conclusions

In all attempts, the Hearth FHIR server has better performance than the HAPI FHIR server. Creation and update actions take a lot of more time in the HAPI FHIR than in the Hearth. The search actions have better results but still, Hearth server is faster about 4 times. I noticed that the HAPI FHIR server caches results (not know for how long) for search requests:

hapi_1                   | 2018-09-05 13:42:12.190 [http-nio-8080-exec-7] INFO  c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:151] Search entity marked as finished
hapi_1                   | 2018-09-05 13:42:12.217 [http-nio-8080-exec-5] INFO  c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:301] Reusing search f36b5bb4-d58c-4836-83fe-448225372c85 from cache
hapi_1                   | 2018-09-05 13:42:12.228 [http-nio-8080-exec-5] INFO  c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:151] Search entity marked as finished
hapi_1                   | 2018-09-05 13:42:12.245 [http-nio-8080-exec-8] INFO  c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:301] Reusing search f36b5bb4-d58c-4836-83fe-448225372c85 from cache
hapi_1                   | 2018-09-05 13:42:12.250 [http-nio-8080-exec-8] INFO  c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:151] Search entity marked as finished
hapi_1                   | 2018-09-05 13:42:12.265 [http-nio-8080-exec-1] INFO  c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:301] Reusing search f36b5bb4-d58c-4836-83fe-448225372c85 from cache
hapi_1                   | 2018-09-05 13:42:12.269 [http-nio-8080-exec-1] INFO  c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:151] Search entity marked as finished
hapi_1                   | 2018-09-05 13:42:12.288 [http-nio-8080-exec-4] INFO  c.u.f.j.s.SearchCoordinatorSvcImpl [SearchCoordinatorSvcImpl.java:301] Reusing search f36b5bb4-d58c-4836-83fe-448225372c85 from cache

Performance results

The following zip file contains all result data of all attempts

  • No labels