When multiple services are deployed in a virtual machine it will be necessary to include a piece in the architecture that guarantees to expose and redirect traffic to them easily and safely.
This article is part of the Microservices Workshop series where the details of the technologies used in the Cinema use case are exposed.
Traefik Proxy
Traefik Proxy is an Edge Router, a modern reverse proxy and load balancer designed to deploy microservices easily and quickly. Among the main characteristics of Traefik is its speed and ease of configuration.
In the Cinema use case, the Traefik Proxy service appears with the name proxy. The complete description of it is in the docker-compose.yml file.
version: "3.8"
services:
proxy:
image: traefik:v2.4.2
command:
# - "--log.level=DEBUG"
- "--api"
- "--api.dashboard"
- "--api.insecure" # Don't do that in production
- "--providers.docker"
- "--entrypoints.web.address=:80"
volumes:
- type: bind
source: /var/run/docker.sock
target: /var/run/docker.sock
read_only: true
...
# see the complete code in https://github.com/mmorejon/microservices-docker-go-mongodb
In Traefik there are multiple providers, eg: Kubernetes Ingress, ECS, Rancher, among others. Each one has specific configurations according to your needs. In this previous snippet you can identify that the Docker provider has been used.
The use of port 80 has been defined as the entry point to the rest of the services and the Docker socket as volume /var/run/docker.sock to dynamically discover the services.
During the start-up of Traefik, it will be necessary to pay attention to the logs to know if the system responds as expected. The logs in DEBUG mode can be enabled through the line "--log.level=DEBUG". Then use the following Docker Compose command to track events.
$ docker-compose logs -f proxy
Attaching to microservices-docker-go-mongodb_proxy_1
proxy_1 | time="2021-02-16T10:34:51Z" level=info msg="Configuration loaded from flags."
proxy_1 | time="2021-02-16T10:34:51Z" level=info msg="Traefik version 2.4.2 built on 2021-02-02T17:20:41Z"
proxy_1 | time="2021-02-16T10:34:51Z" level=debug msg="Static configuration loaded {\"global\":{\"checkNewVersion\":true},\"serversTransport\":{\"maxIdleConnsPerHost\":200},\"entryPoints\":{\"traefik\":{\"address\":\":8080\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{},\"http\":{}},\"web\":{\"address\":\":80\",\"transport\":{\"lifeCycle\":{\"graceTimeOut\":10000000000},\"respondingTimeouts\":{\"idleTimeout\":180000000000}},\"forwardedHeaders\":{},\"http\":{}}},\"providers\":{\"providersThrottleDuration\":2000000000,\"docker\":{\"watch\":true,\"endpoint\":\"unix:///var/run/docker.sock\",\"defaultRule\":\"Host(`{{ normalize .Name }}`)\",\"exposedByDefault\":true,\"swarmModeRefreshSeconds\":15000000000}},\"api\":{\"insecure\":true,\"dashboard\":true},\"log\":{\"level\":\"DEBUG\",\"format\":\"common\"}}"
...
Will the details of the Traefik configuration be available through the UI?
Traefik Dashboard
Traefik Dashboard will allow you to centrally view the components of each service defined in the Compose file. Entrypoints, Routers, Middlewares y Services are some of the main elements that you will be able to identify in the dashboard. You will need to pay particular attention to these components to achieve the correct traffic between the pieces of software.
You will be able to access into Traefik dashboard once the processes have started as indicated in the Cinema use case. The following image corresponds to the Traefik dashboard deployed.
Access the Routers section so that you know all the inbound rules enabled for the services.
Enable access to internal APIs
For security reasons the only exposed service of the Cinema application is the website. All other microservices are restricted to the subnet created by Docker Compose.
To show the concepts that you will see below, it will be necessary to enable access to the internal APIs. Traefik allows this configuration to be carried out easily through labels. The next two labels will be responsible for defining the path and port to a microservices.
...
labels:
# Enable public access
- "traefik.http.routers.movies.rule=PathPrefix(`/api/movies/`)"
- "traefik.http.services.movies.loadbalancer.server.port=4000"
...
# see the complete code in https://github.com/mmorejon/microservices-docker-go-mongodb
Enable these labels in the users, movies, showtimes and bookings services. Then use the following command to restart the services.
$ docker-compose up -d
Access the Traefik Dashboard again and observe the new routes included in the list.
Traefik Middlewares
What are Middlewares and what do I need them for?
Middleware (middle - software, software in the middle) will allow you to include functionalities and behaviors to your services through configurations. Traefik has the Middleware concept, where you can concatenate multiple pieces of software before traffic reaches the destination service. See official documentation.
It is important that you master the Traefik Middleware concept because it will save you development time and flexibility in your configurations. A first step to understand it will be to go to the Middleware section of the dashboard. The two items that you will see in the list are created by Traefik to guarantee access to the dashboard through the URL /dashboard.
How to include the available Traefik middlewares in the services defined in Compose?
To include a middleware in a Docker Compose service, it will be necessary to define it and then assign it. It is important to note that a definition can be assigned to multiple services.
Define Middlewares
The definition and assignment of middlewares is done through labels. In the docker-compose.yml file, three elements have been added that can be activated if you delete the comment at the beginning of each line. Each label corresponds to an existing middleware in Traefik.
services:
proxy:
image: traefik:v2.4.2
command:
...
labels:
# Create basics middlewares
- "traefik.http.middlewares.my-auth.basicauth.users=test:$$apr1$$H6uskkkW$$IgXLP6ewTrSuBkTrqE8wj/"
- "traefik.http.middlewares.my-compress.compress=true"
- "traefik.http.middlewares.my-header.headers.customresponseheaders.X-Custom-Response-Header=cinema"
...
# see the complete code in https://github.com/mmorejon/microservices-docker-go-mongodb
Restart the proxy service and check the dashboard items list again.
$ docker-compose up -d
...
Recreating microservices-docker-go-mongodb_proxy_1 ...
Recreating microservices-docker-go-mongodb_proxy_1 ... done
Identify the three middleware added in the listing. Now these new components are ready to be assigned to services.
Assign Middlewares to a service
The assignment of middleware to services is done through labels. Here’s how to include the new middlewares in the movies service flow.
At the moment the service is defined as follows:
movies:
image: ghcr.io/mmorejon/cinema-movies:v2.1.0
command:
- "-mongoURI"
- "mongodb://db:27017/"
labels:
- "traefik.http.routers.movies.rule=PathPrefix(`/api/movies/`)"
- "traefik.http.services.movies.loadbalancer.server.port=4000"
...
# see the complete code in https://github.com/mmorejon/microservices-docker-go-mongodb
The labels that exist at this time tell Traefik the URL and port to use to access the service. In the movies service view of the dashboard you can see the stream in use. You will notice that there are no assigned middleware.
The next step will be to include the label that allows assigning multiple middlewares to the movies service router.
movies:
image: ghcr.io/mmorejon/cinema-movies:v2.1.0
command:
- "-mongoURI"
- "mongodb://db:27017/"
labels:
- "traefik.http.routers.movies.rule=PathPrefix(`/api/movies/`)"
- "traefik.http.services.movies.loadbalancer.server.port=4000"
# Apply the middleware
- "traefik.http.routers.movies.middlewares=my-header,my-compress"
...
# see the complete code in https://github.com/mmorejon/microservices-docker-go-mongodb
Restart the movies service to include the new configuration.
$ docker-compose up -d
...
Recreating microservices-docker-go-mongodb_movies_1 ...
Recreating microservices-docker-go-mongodb_movies_1 ... done
Refresh the dashboard page and you can see the middlewares as part of the movies service flow.
Check its correct behavior using the following command.
$ curl -X GET -I http://localhost/api/movies/
HTTP/1.1 200 OK
Content-Length: 783
Content-Type: application/json
Date: Tue, 16 Feb 2021 14:23:27 GMT
Vary: Accept-Encoding # <-- my-compress middleware
X-Custom-Response-Header: cinema # <-- my-header middleware
If you wish, you can assign these functionalities to the rest of the services that need them without having to define them again.
The new functionalities included through the middleware are of great importance for traffic between services, but another equally important topic has not yet been seen: metrics.
Is it possible to view the metrics of the deployed services on the dashboard?
Traefik Pilot
Traefik Pilot is a platform as a service (SaaS) conceived to extend Traefik’s functionalities and include visibility through metrics (Observability).
To connect the dashboard with this platform, you must click the Connect with Traefik Pilot button located in the upper right corner of the screen. Then select one of the alternatives that appear to complete the registration.
The option used to carry out this example was GitHub.
Once registered, a screen is displayed where your user appears along with the instructions to include the instance you are using. In this case the instance is the proxy service defined in the docker-compose.yml file.
The next step will be to use the Register Current Traefik Instance button to obtain the token that will allow associating the proxy service as an instance of the Pilot platform.
Look in the proxy service of the docker-compose.yml file for the line --pilot.token= and replace the value with the token generated by Pilot.
proxy:
image: traefik:v2.4.2
command:
- "--log.level=DEBUG"
- "--api=true"
- "--api.dashboard=true"
- "--api.insecure=true" # Don't do that in production
- "--providers.docker"
- "--entrypoints.web.address=:80"
# add pilot token service
- "--pilot.token=3d1bd97f-cb9d-4cde-8112-77ce2f5abc92"
...
# see the complete code in https://github.com/mmorejon/microservices-docker-go-mongodb
Then restart the proxy service with the docker-compose up -d command and access the dashboard again so you can learn more about the Cinema application metrics.
You may have noticed a Plugins tab below Metrics, so you are probably wondering:
What are Plugins in Traefik and how can I use them?
Traefik Plugins
Plugins are external functionalities to the Traefik core that allow enriching the system with new behaviors. Most are community-developed, which means you can use existing ones or create a new one.
At this time there are multiple plugins that can be included quickly and easily.
The plugin selected to use as an example is Rewrite Body. The mission of this plugin is to modify the content of the message response.
The first thing you will need to do is include the plugin in the proxy service settings so that it downloads this functionality during system startup. It will then go on to register the plugin as a middleware via the Docker Compose labels. The following snippet shows how the proxy service looks:
proxy:
image: traefik:v2.4.2
command:
...
# add pilot token service
- "--pilot.token=3d1bd97f-cb9d-4cde-8112-77ce2f5abc92"
# install plugin rewrite body
- "--experimental.plugins.plugin-rewrite.modulename=github.com/traefik/plugin-rewritebody"
- "--experimental.plugins.plugin-rewrite.version=v0.3.1"
labels:
# Create basics middlewares
- "traefik.http.middlewares.my-auth.basicauth.users=test:$$apr1$$H6uskkkW$$IgXLP6ewTrSuBkTrqE8wj/"
- "traefik.http.middlewares.my-compress.compress=true"
- "traefik.http.middlewares.my-header.headers.customresponseheaders.X-Custom-Response-Header=cinema"
# Create middlewares for plugins
- "traefik.http.middlewares.my-rewrite.plugin.plugin-rewrite.lastModified=true"
- "traefik.http.middlewares.my-rewrite.plugin.plugin-rewrite.rewrites[0].regex=ID" # <-- valor a modificar
- "traefik.http.middlewares.my-rewrite.plugin.plugin-rewrite.rewrites[0].replacement=MY-ID" # <-- nuevo valor
...
# see the complete code in https://github.com/mmorejon/microservices-docker-go-mongodb
Restart the service and check that the plugin has been registered as middleware correctly.
The last step will be to assign the plugin middleware to the movies service, but before doing so list the existing data without modifications.
$ curl -X GET http://localhost/api/movies/
[{"ID":"6002130f0f646161d37f7960","Title":"The Lighthouse","Director":"Robert Eggers","Rating":4.4,"CreatedOn":"0001-01-01T00:00:00Z"},..
Note: If the information you get is empty
[ ]your will need to restore the data in the Cinema application.
Now include the new middleware in the existing movies service listing.
...
# Apply the middleware
- "traefik.http.routers.movies.middlewares=my-header,my-compress,my-rewrite"
...
# see the complete code in https://github.com/mmorejon/microservices-docker-go-mongodb
Restart the movies service and query again. You will notice that ID has been replaced by MY-ID.
$ curl -X GET http://localhost/api/movies/
[{"MY-ID":"6002130f0f646161d37f7960","Title":"The Lighthouse","Director":"Robert Eggers","Rating":4.4,"CreatedOn":"0001-01-01T00:00:00Z"},...
Conclusions
Traefik Labs keeps constantly evolving and adapting to the new needs of the industry. This article has shown the options that Traefik provides to include visibility, metrics and functionalities to the services managed from Docker Compose.
Remember to deep into the concepts of Middleware, Pilot and Plugins to get the most benefit from the platform.
comments powered by Disqus
