09.26.19

Elastic Kubernetes Service (EKS) Observability with Istio Service Mesh

By JP La Torre
Elastic Kubernetes Service (EKS) Observability with #IstioServiceMesh

Secure, reliable, and observable. These three objectives are what systems engineers aim for when developing a solution, both from an infrastructure standpoint and from a service design perspective. Creating a solution that is secure, reliable, and observable, however, is easier said than done.

Over the years, there have been many innovations that make achieving these goals easier. The rapid growth of Kubernetes as a platform, for example, makes creating a complex environment that hosts microservices a lot easier.

Even services like Amazon Web Services are becoming more robust. The introduction of Elastic Kubernetes Service (EKS) is a step in the right direction for developers. Amazon EKS isn’t just robust, but also highly available and observable.

Istio Service Mesh makes achieving those objectives even easier. The all-in-one framework certainly makes connecting, securing, controlling, and observing services simpler.

An Open-Source Mesh

Istio is open source. It is designed from the ground up to function as a service mesh, with service-to-service communication being its primary feature. Developers no longer have to go through the process of setting up service-to-service connections that are reliable and highly available. More importantly, there is no need to deal with the hassle of maintaining such complex connections.

Istio Service Mesh cannot be seen as a mesh of microservices. It is more of a Layer 7 proxies creating a network through which microservices can communicate with each other. Istio is unique and familiar at the same time. For instance, you still have a data plane and a control plane for the mesh.

The data plane is designed to work with Kubernetes and EKS natively. It can be deployed as a sidecar to each container. Once deployed, the data plane handles everything related to service-to-service communications.

The control plane handles the bigger picture. It is a separate part of the Istio Service Mesh, meaning proxies already implemented as sidecars don’t need to constantly be in contact with the control plane. It is also a basic proxy for the entire mesh.

The control plane of an Istio Service Mesh consists of three components. The component that handles configurations is aptly named Pilot. This is the component that defines routing rules and allows for proxies within the mesh to communicate with each other.

The second component is Mixer, which handles the collection of key metrics from traffic, including logging and connecting to monitoring systems. Mixer also takes care of handling queries and requests from the data plane.

Lastly, we have Citadel, which turns Istio Service Mesh into a powerful foundation for zero-trust environments. Each service can have its own certificate and the need for network controls can be eliminated entirely.

Istio Service Mesh for Higher Observability

The component we are going to focus on the most in this article is Mixer, which turns Istio into the perfect framework for better Kubernetes observability. The component is specifically designed to bring several benefits regardless of how microservices are designed to work with each other.

For starters, Mixer automatically pulls monitoring metrics, so you don’t need to perform individual pull requests to microservices to collect the necessary metrics. This alone is a huge boost to observability since you only need to collect key metrics from Istio’s control plane.

While pulling key metrics, Mixer also handles second-level caching for sidecars. It improves latency and makes sure that the entire system can continue to be scalable. Combined with a well-designed set of microservices, achieving high availability also becomes easier.

Backend abstraction is another big benefit offered by Mixer and Istio as a whole. The service mesh handles policy controls as well as detailed telemetry collection. Istio works natively with EKS and Amazon’s own monitoring suite.

To complete the set, Mixer provides ways to interact with databases. If you want to measure the performance of your system in a more comprehensive way, this feature of refined interactions with databases is a huge plus.

Gaining Insights

High observability is only the beginning. When monitoring is performed with the help of Istio, there are several important insights that can be acquired in the process. I personally love how Istio can manage traces and logs for all traffic in a cluster, including ingress and egress.

The comprehensive nature of Istio’s monitoring features allows for granular analysis as well as a holistic overview of the cluster. It also works with custom dashboards, plus it can be deployed using a custom helm template.

The next part of the process is visualizing observability, which is something that Istio is very good at. It works with custom dashboards; I recommend using Kiali to monitor availability and observability in a visual way.

Kiali is simply stunning. You can monitor multiple namespaces, create charts that represent how services communicate with each other, and process key metrics collected by Mixer using filters and other advanced tools. You also get raw data on the sidebar along with time frames to choose from.

Alerts are particularly handy. You can check the health of your cloud cluster, the health of each service, or the efficiency level of microservices. When events like high error rate are detected, Kiali automatically notifies you on the primary source of the problem.

It is worth noting that Istio Service Mesh also works with other data visualization tool, including Grafana and other native add-ons. There are multiple ways to gain insight from Istio, plus you can integrate it with existing Amazon monitoring tools like CloudWatch.

Istio can certainly take the difficulties out of setting up highly observable systems in Kubernetes. However, it’s not recommended for smaller projects perhaps. Resource consumption is significant just because you will need to add more or bigger nodes to the cluster—since it adds an envoy proxy as a sidecar on each pod, plus a pilot and all the other resources necessary. If you’re running smaller projects, then it may be worth considering an alternative over the extra resource consumption. Check out our service mesh comparison article here for some further advice on the subject.

Istio Service Mesh is a fantastic addition to any cluster that hosts multiple services. But it’s always worth weighing up the pros and cons of any tool before adoption and ensuring it’s a fit for your business-critical workloads and not just being chosen, say, for its popularity.


Caylent provides a critical DevOps-as-a-Service function to high growth companies looking for expert support with Kubernetes, cloud security, cloud infrastructure, and CI/CD pipelines. Our managed and consulting services are a more cost-effective option than hiring in-house, and we scale as your team and company grow. Check out some of the use cases, learn how we work with clients, and read more about our DevOps-as-a-Service offering.