Istio and Service Discovery in K8s

Service discovery in Kubernetes is a mechanism that allows applications and services to find each other and communicate within the Kubernetes cluster. It operates primarily through two main components: Services and DNS. Here’s how it works:

1. Kubernetes Services

Service Resource: A Kubernetes Service is an abstraction that defines a logical set of Pods and a policy to access them. When a Service is created, it gets an IP address and a DNS name.
Types of Services: There are different types of Services in Kubernetes, such as ClusterIP (default), NodePort, LoadBalancer, and ExternalName, each serving different use cases.
Label Selector: Services use label selectors to identify the group of Pods they provide access to. Any Pod in the cluster that matches the selector will be part of the Service.

2. DNS for Service Discovery

Kubernetes DNS: When a Service is created in Kubernetes, a DNS record is also automatically created for that Service. The default DNS service in Kubernetes is CoreDNS.
DNS Naming: Services are typically reachable via a DNS name like service-name.namespace.svc.cluster.local, where service-name is the name of the Service, namespace is the Kubernetes namespace, and svc.cluster.local is the default domain.
DNS Resolution: When a Pod tries to access a Service, it uses the Service’s DNS name. DNS resolves the name to the Service’s ClusterIP, and traffic is then routed to one of the Pods that are part of the Service.

3. Environment Variables

Service Information as Environment Variables: Kubernetes also injects information about Services into Pod containers as environment variables. This provides another way for applications to discover Services.

4. Load Balancing

Internal Load Balancing: When multiple Pods are part of a Service, the Service acts as a load balancer, distributing network traffic among these Pods. This load balancing is typically round-robin.

5. Service Endpoints

Endpoints Object: Kubernetes maintains an Endpoints object for each Service, which keeps track of the IP addresses of the Pods that are included in the Service.

6. Headless Services

Direct Pod Access: For some use cases, like when you need to have direct access to individual Pods, you can use a headless Service. This type of Service doesn’t have a ClusterIP, and the DNS system will return the Pod IPs directly.

Service discovery in Kubernetes simplifies the process of connecting applications running in a distributed and dynamic environment. By using Services and DNS, applications can easily locate and communicate with each other without hard-coding specific IP addresses, making the system more resilient and scalable.

Istio is an open-source service mesh that provides a way to control how microservices share data with one another. It’s often used in Kubernetes (K8s) environments, but it’s not limited to them.

What is Istio?

Service Mesh: Istio is a service mesh, which is a dedicated infrastructure layer for handling service-to-service communication. It’s primarily used to make network communication between microservices more secure, fast, and reliable.
Components: Istio consists of a data plane and a control plane. The data plane is typically comprised of a set of intelligent proxies (Envoy) deployed as sidecars. These proxies mediate and control all network communication between microservices. The control plane manages and configures the proxies to route traffic.
Capabilities: It provides several key features like traffic management, security, and observability. It allows you to define rules that control how microservices communicate with each other, enforce policies, and aggregate telemetry data.

The architecture of Istio has evolved to become more streamlined and efficient, primarily focusing on a single control plane component called Istiod. This simplification has made Istio easier to install, manage, and operate. Here are the key components/objects in the latest versions of Istio and their functions:

1. Istiod

Unified Control Plane: Istiod is a unified control plane that combines the functionality of several older components (Pilot, Citadel, Galley, and the sidecar injector).
Service Discovery: It manages service discovery, providing information about the services in the mesh, including their locations.
Configuration Management: Istiod is responsible for distributing service mesh configuration to the Envoy proxies.
Certificate Management: It automates the key and certificate rotation for Istio workloads, ensuring secure communication.
Sidecar Injection: Istiod automatically injects Envoy sidecar proxies into Kubernetes pods.

2. Envoy Proxy (Sidecar)

Data Plane Component: The Envoy proxy is a high-performance proxy that mediates all inbound and outbound traffic for all services in the service mesh.
Traffic Management: It handles dynamic service discovery, load balancing, TLS termination, HTTP/2 & gRPC proxying, and more.
Resilience and Observability: Provides features like circuit breaking, fault injection, and rich metrics.

3. Custom Resource Definitions (CRDs)

Gateway: Defines a load balancer for HTTP/TCP traffic, most often used for ingress but sometimes also for egress.
VirtualService: Provides routing rules to manage traffic within the service mesh.
DestinationRule: Defines policies applied to traffic after VirtualService routing, like load balancing, circuit breakers.
ServiceEntry: Adds an entry to Istio’s internal service registry, allowing Istio to route to services outside of the mesh.
PeerAuthentication: Defines mutual TLS settings at the global, namespace, or workload level.
AuthorizationPolicy: Specifies access control policies.

4. Ingress and Egress Gateways

Managed Traffic Entry and Exit: These are special Envoy proxies that manage the entry and exit of traffic to and from the mesh.
Security and Monitoring: They provide a secure way to manage ingress and egress traffic, and allow for monitoring and logging.

5. Add-ons (Optional)

Prometheus: For capturing metrics.
Grafana: Provides visualizations for Istio metrics.
Jaeger/Zipkin: For tracing and monitoring.
Kiali: Offers service mesh observability and configuration.

The Istio architecture aims to provide a comprehensive, yet simplified solution for managing the complex interactions and policies of services in a Kubernetes environment. It focuses on ease of use, security, and performance, making it a popular choice for implementing service mesh in modern cloud-native applications.

(image from: https://istio.io/v1.16/about/service-mesh/)

Understanding service discovery in Kubernetes and its interaction with a service mesh like Istio requires knowledge of both native Kubernetes mechanisms and the additional capabilities provided by Istio.

Native Kubernetes Service Discovery

Kubernetes natively supports service discovery using the following concepts:

Pods: The smallest deployable units in Kubernetes, which can run one or more containers.
Labels and Selectors: Pods are labeled with key-value pairs. Services use selectors to select a group of pods based on their labels.
Services: Kubernetes Services are abstractions that define a logical set of Pods and a policy to access them. A Service has an IP address and port that, unlike individual Pods, remain constant.
DNS for Service Discovery: Kubernetes runs a DNS pod and service for cluster-internal DNS. When a Service is created, it gets a DNS entry. This allows Pods to perform service discovery automatically using the service’s DNS name.

Example: ServiceA Communicating with ServiceB

ServiceA sends a request to ServiceB using the DNS name, like serviceb.default.svc.cluster.local.
The DNS name resolves to the cluster IP of ServiceB.
The request is routed to one of the Pods selected by ServiceB based on load-balancing.

Service Discovery with Istio

When Istio is implemented in a Kubernetes cluster, it enhances the existing service discovery mechanism:

Istiod (Control Plane): Acts as the central component managing the configuration and state of the service mesh.
Envoy Sidecars: Each pod gets an Envoy sidecar proxy that intercepts all inbound and outbound traffic.

Example: ServiceA Communicating with ServiceB via Istio

ServiceA sends a request to ServiceB.
Instead of going directly to ServiceB, the request is intercepted by ServiceA’s Envoy sidecar.
The sidecar proxy queries Istiod for the location and health of ServiceB‘s endpoints.
Istiod responds with the appropriate endpoints of ServiceB.
The sidecar then forwards the request to one of these endpoints, typically another sidecar proxy in front of ServiceB‘s Pod.

What Happens If ServiceB’s Pods Disappear?

Kubernetes Behavior: The endpoints of ServiceB are updated, and any requests to ServiceB would fail to find a valid endpoint.
Istio’s Response:
The Envoy sidecars are constantly informed by Istiod about the state of the service mesh, including the availability of service endpoints.
If ServiceB‘s Pods disappear, Istiod updates the sidecars with this information.
ServiceA’s sidecar, upon not finding healthy endpoints for ServiceB, can apply resilience features like retries, circuit breaking, or failing over to another service as per the configured rules.
This dynamic update and advanced traffic management capabilities of Istio help in maintaining the stability and reliability of the service communication.

Comunications Between isitod and Envoy Sidecars

Istiod communicates with the Envoy sidecars in an Istio service mesh using a combination of event-driven updates and periodic polling, depending on the situation. The process relies on the xDS (eXtensible Discovery Service) API, which Envoy uses to dynamically discover network resources. Here’s how it works:

xDS API in Envoy

xDS API: Envoy uses the xDS API for dynamic resource discovery. This API includes various discovery services like LDS (Listener Discovery Service), CDS (Cluster Discovery Service), EDS (Endpoint Discovery Service), and RDS (Route Discovery Service).

Istiod communicates with Envoy sidecars using the xDS API over a gRPC stream. This communication is primarily event-driven, reacting to changes in the service mesh, but also includes periodic polling to ensure configuration consistency.

Istio as a Service Discovery Component in Kubernetes

Kubernetes provides a robust native service discovery mechanism based on DNS and Services. Istio enhances this by introducing a service mesh layer that enables advanced traffic management, resilience features, and real-time updates on service availability, providing a more dynamic, secure, and efficient communication process between services like ServiceA and ServiceB.

Integration with Kubernetes: In a Kubernetes environment, Istio integrates seamlessly, extending its functionalities. Kubernetes itself provides service discovery and load balancing for services running on it. Istio complements these features.
Service Discovery: Istio uses the services registered with Kubernetes for service discovery. When a service in Kubernetes is created, Istio will automatically detect it and use it in its routing rules. This is crucial in a microservices architecture where knowing the location of a service is essential for communication.
Load Balancing and Traffic Routing: Through its intelligent routing capabilities, Istio can control the flow of traffic and API calls between services. This includes advanced load balancing across multiple instances of a service.
Dynamic Updates: As services are scaled up or down in Kubernetes, Istio dynamically adjusts its routing rules. This means that the service discovery component is always up-to-date with the current state of the Kubernetes cluster.
Fault Injection and Traffic Shaping: Istio can also simulate faults or changes in network behavior, which helps in testing the resilience of the system. It can also shape the traffic based on various criteria, aiding in blue/green deployments, A/B testing, etc.

Istio automatically detects new services in a Kubernetes (K8s) environment through its integration with Kubernetes’ service discovery mechanisms.

Kubernetes Service Discovery Basics

Kubernetes Services: In Kubernetes, a ‘Service’ is an abstraction which defines a logical set of Pods (the smallest deployable units in Kubernetes) and a policy by which to access them. Services in Kubernetes serve as a key part of its internal service discovery mechanism.
Endpoints: Kubernetes maintains a list of ‘Endpoints’, which are IP addresses and ports where Pods can be reached. These Endpoints are updated whenever Pods are added, removed, or changed.
Labels and Selectors: Services are linked to Pods through labels and selectors. A service’s selector matches labels on Pods to include them in its set.

Istio’s Integration with Kubernetes Service Discovery

Watching the Kubernetes API Server: Istio’s control plane components, specifically the Pilot (or Istiod in newer versions), keep a watch on the Kubernetes API server. They listen for changes in services, pods, and endpoints.
Automatic Detection: When a new service is created in Kubernetes, it’s registered with the Kubernetes API server. Istio’s control plane detects this new service by watching the API server.
Service and Endpoint Information: Istio retrieves information about the service and its endpoints (i.e., the IP addresses and ports of the Pods that form the service) from Kubernetes.
Updating Envoy Proxies: After detecting a new service and its associated endpoints, Istio’s control plane updates the Envoy proxies (which run as sidecars to each service’s Pod). It provides them with the latest routing information, ensuring that traffic can be correctly routed to the new service.
Dynamic Configuration: This process is dynamic and happens automatically as services are added, updated, or removed in Kubernetes. Istio ensures that the Envoy proxies always have up-to-date information to properly route requests.
Service Mesh Configuration: In addition to basic service discovery, Istio allows you to define more advanced traffic management rules, such as canary deployments, A/B testing, or circuit breakers, and applies these configurations dynamically as well.

Istio provides a powerful and dynamic environment for managing microservices communication in a Kubernetes cluster. This seamless integration is one of the reasons why Istio is a popular choice for service mesh implementation in Kubernetes environments.

When using Istio, or specifically the Envoy sidecars that Istio deploys with each service in a Kubernetes (K8s) environment, the communication between Service A and Service B is managed and facilitated by these sidecars. Here’s an overview of how this communication process works and how Service A knows about the connectivity and the nodes of Service B:

Basic Communication Process

Deployment of Envoy Sidecars: In an Istio-enabled K8s cluster, each pod typically has an Envoy proxy sidecar. This sidecar intercepts all incoming and outgoing network traffic to the pod.
Service A Initiates Communication: When Service A wants to communicate with Service B, it sends a request. Instead of sending this request directly to Service B, it sends it to its own Envoy sidecar.
Service Discovery by Istio: The sidecar uses the service discovery information provided by Istio, which is aware of all services in the K8s cluster, their endpoints, and their current state.
Routing Decision: The Envoy sidecar of Service A uses Istio’s dynamic service discovery and routing rules to determine where to send the request. It selects the appropriate instance of Service B based on the load balancing and routing configuration.
Forwarding the Request: The request is then forwarded to the chosen instance of Service B, specifically to the Envoy sidecar of that instance.
Reaching Service B: The Envoy sidecar of Service B then passes the request to the actual Service B application.

Service Availability and Load Information

Health Checks: Envoy proxies can be configured to perform health checks on services. These checks determine whether an instance of a service is healthy and can handle requests.
Load Balancing: Envoy sidecars use various algorithms for load balancing. They are aware of each instance of Service B and their health status. This information is used to intelligently distribute the load.
Dynamic Updates: Istio’s control plane (Istiod) monitors the K8s API for changes in services, pods, and their health statuses. It updates the Envoy proxies with this information, ensuring they have the latest data about the availability and health of Service B.
Circuit Breaking: Istio can be configured to implement circuit breaking patterns. If Service B becomes overloaded or starts failing, the circuit breaker can trip, preventing further requests to Service B, ensuring system stability.
Resilience Features: Features like retries, timeouts, and failovers are also managed by Istio, providing additional mechanisms to ensure that communication is resilient and reliable.

Read more paid content for interview questions about istio and k8s below.

Subscribe to get access

Read more of this content when you subscribe today.