Working around OpenLMIS Reference Distribution container registration failures
What is the problem?
On some computers with Ubuntu OS, services sometimes happen to register themselves to Consul with improper host (127.0.0.1), which causes Consul being unable to properly redirect requests, as it does not see their real IP addresses. You can verify that is the case by entering Consul Web UI, and checking services' addresses (those improperly registered will show 127.0.0.1 as their registered host). This may happen randomly to any service, and sometimes needs several restarts to have all of them working properly. As mentioned above, the problem occurs on some computers with Ubuntu (14 or 16) installed, though there's still more investigation needed to see what causes the issue.
Temporary solution:
This is a temporary solution to speed up booting process while the problem is still investigated.
Usually, only one (or few) services are registered inproperly, so we don't need to restart whole OpenLMIS Reference Distribution instance, we can restart only "corrupted" services instead:
1. See which services are registered on localhost (you can do this via consul ui, or `docker-compose exec nginx cat /etc/nginx/conf.d/default.conf`- under upstreams section).
(next steps for each service registered on localhost)
2. Run `docker-compose exec service_name node consul/registration.js -c deregister -f consul/config.json -r consul/api-definition.yaml` - to deregister the container on consul
(for refui: `docker-compose exec service_name node consul/registration.js -c deregister -f consul/config.json`)
3. Run `docker-compose kill service_name && docker-compose rm -f service_name` (or `docker rm -f service_container_name`) - to remove the container
4. Re-run service, by `docker-compose up service_name -d`
For steps 2, 3 and 4, I've made a script to make the thing shorter (the file is attached to this page).
Usage:
./restart_service <container_names>
Example:
./restart service auth requisition referencedata
OpenLMIS: the global initiative for powerful LMIS software