All things about Istio

Istio is an open-source service mesh that provides a way to control how microservices share data and resources. It’s designed to handle the complexity of network management inherent in microservice architectures, providing key functionalities like service discovery, load balancing, encryption, observability, and policy enforcement.

How Istio Works:

1. Sidecar Pattern: Istio uses a sidecar model with the Envoy proxy. Each microservice in the mesh is paired with an Envoy proxy that handles all network communication. This means the microservices don’t need to implement any specific logic for inter-service communication.
2. Control Plane and Data Plane: Istio’s architecture is divided into two main components: the control plane and the data plane.
   • Control Plane: Responsible for managing and configuring proxies to route traffic.
   • Data Plane: Made up of a set of intelligent proxies (Envoy) deployed as sidecars that interrupt and control all network communication between microservices.

Key Components of Istio:

1. Envoy Proxy:
   • The default proxy used by Istio.
   • Mediates all inbound and outbound traffic for all services in the service mesh.
   • Provides dynamic service discovery, load balancing, TLS termination, HTTP/2 & gRPC proxying, circuit breakers, health checks, staged rollouts with %-based traffic split, and fault injection.
2. Pilot:
   • Provides service discovery for the Envoy sidecars, traffic management capabilities for intelligent routing (A/B testing, canary deployments), and resiliency (timeouts, retries, circuit breakers).
   • Translates high-level routing rules that control traffic behavior into Envoy-specific configurations.
3. Citadel:
   • Provides strong service-to-service and end-user authentication with built-in identity and credential management.
   • Enables in-service mesh authentication and identity management.
4. Galley:
   • Responsible for validating, processing, and distributing config states to the other components of Istio.
   • Ensures the configurations applied to the service mesh are valid and consistent.
5. Mixer:
   • Handles policy decisions in the service mesh.
   • Enforces access control and usage policies across the service mesh, and collects telemetry data from the Envoy proxy and other services.

When a service in the mesh needs to communicate with another service, the Envoy sidecar proxy intercepts the traffic and routes it based on the rules configured in the control plane.The control plane, consisting of Pilot, Citadel, and Galley, manages and configures the proxies to route traffic, enforce policies, and aggregate telemetry data.Mixer (which is being deprecated in favor of newer approaches like Istio Telemetry V2) provides policy controls and telemetry collection.

The architecture of Istio has evolved to become more streamlined and efficient, primarily focusing on a single control plane component called Istiod. This simplification has made Istio easier to install, manage, and operate. Here are the key components/objects in the latest versions of Istio and their functions:

1. Istiod

• Unified Control Plane: Istiod is a unified control plane that combines the functionality of several older components (Pilot, Citadel, Galley, and the sidecar injector).
• Service Discovery: It manages service discovery, providing information about the services in the mesh, including their locations.
• Configuration Management: Istiod is responsible for distributing service mesh configuration to the Envoy proxies.
• Certificate Management: It automates the key and certificate rotation for Istio workloads, ensuring secure communication.
• Sidecar Injection: Istiod automatically injects Envoy sidecar proxies into Kubernetes pods.

2. Envoy Proxy (Sidecar)

• Data Plane Component: The Envoy proxy is a high-performance proxy that mediates all inbound and outbound traffic for all services in the service mesh.
• Traffic Management: It handles dynamic service discovery, load balancing, TLS termination, HTTP/2 & gRPC proxying, and more.
• Resilience and Observability: Provides features like circuit breaking, fault injection, and rich metrics.

3. Custom Resource Definitions (CRDs)

• Gateway: Defines a load balancer for HTTP/TCP traffic, most often used for ingress but sometimes also for egress.
• VirtualService: Provides routing rules to manage traffic within the service mesh.
• DestinationRule: Defines policies applied to traffic after VirtualService routing, like load balancing, circuit breakers.
• ServiceEntry: Adds an entry to Istio’s internal service registry, allowing Istio to route to services outside of the mesh.
• PeerAuthentication: Defines mutual TLS settings at the global, namespace, or workload level.
• AuthorizationPolicy: Specifies access control policies.

4. Ingress and Egress Gateways

• Managed Traffic Entry and Exit: These are special Envoy proxies that manage the entry and exit of traffic to and from the mesh.
• Security and Monitoring: They provide a secure way to manage ingress and egress traffic, and allow for monitoring and logging.

5. Add-ons (Optional)

• Prometheus: For capturing metrics.
• Grafana: Provides visualizations for Istio metrics.
• Jaeger/Zipkin: For tracing and monitoring.
• Kiali: Offers service mesh observability and configuration.

The Istio architecture aims to provide a comprehensive, yet simplified solution for managing the complex interactions and policies of services in a Kubernetes environment. It focuses on ease of use, security, and performance, making it a popular choice for implementing service mesh in modern cloud-native applications.

(image from: https://istio.io/v1.16/about/service-mesh/)

Benefits of Using Istio:

• Traffic Management: Fine-grained control over traffic with rich routing rules, retries, failovers, and fault injection.
• Security: Strong identity-based authentication and authorization.
• Observability: Detailed insights into service behavior with telemetry data.
• Resilience: Enhanced service-to-service communication with retries, timeouts, circuit breakers.
• Platform Agnostic: Works on a variety of environments including Kubernetes, VMs, and various cloud platforms.

Istio’s Ingress Gateway is a crucial component for managing incoming traffic to the services within an Istio service mesh. It allows you to define how external traffic is routed to the various services in your mesh, enabling features like load balancing, authentication, monitoring, and more.

Understanding Istio Ingress Gateway:

1. Gateway Resource: In Istio, a Gateway is a custom resource that configures a load balancer for HTTP/TCP traffic, most commonly at the edge of the mesh to enable ingress traffic.
2. Based on Envoy Proxy: The Ingress Gateway is powered by Envoy proxy, which is designed to handle traffic management and routing.
3. Security and Routing: It provides security (like TLS termination) and routing rules to control how external users access services within your mesh.
4. Decoupling from Services: The Ingress Gateway decouples traffic management from application code, meaning that you can control ingress traffic without altering your services.

Deploying Istio Ingress Gateway:

Assuming you have a Kubernetes cluster and Istio installed, here’s how you can deploy an Istio Ingress Gateway:

1. Verify Istio Installation: Ensure that Istio is installed with the Ingress Gateway component. If you’re using the default Istio installation profiles, the Ingress Gateway is typically installed by default.
2. Deploying the Ingress Gateway: If you need to explicitly deploy the Ingress Gateway, use kubectl with Istio’s configuration files: kubectl apply -f install/kubernetes/istio-demo.yaml
3. Configure the Gateway Resource:
   • Define a Gateway resource in YAML format. This example creates a simple gateway that listens for HTTP traffic on port 80:

   apiVersion: networking.istio.io/v1alpha3
   kind: Gateway
   metadata:
     name: example-gateway
   spec:
     selector:
       istio: ingressgateway  # Use Istio's default Ingress Gateway
     servers:
     - port:
         number: 80
         name: http
         protocol: HTTP
       hosts:
       - "*"

4. Apply the Gateway Configuration:
   • Apply the Gateway configuration to your cluster using kubectl: kubectl apply -f example-gateway.yaml
5. Define Routing with VirtualService:
   • To route incoming traffic through the gateway to your services, define a VirtualService:

   apiVersion: networking.istio.io/v1alpha3
   kind: VirtualService
   metadata:
     name: myservice
   spec:
     hosts:
     - "myhost.com"  # Host to which the gateway routes traffic
     gateways:
     - example-gateway
     http:
     - route:
       - destination:
           host: myservice  # Kubernetes service name
           port:
             number: 8080

• Apply it using kubectl: kubectl apply -f myservice-virtualservice.yaml

By properly setting up Istio’s Ingress Gateway, you can effectively manage how external traffic is routed to your services, enhancing the security and efficiency of your service mesh.

Kubernetes Service Discovery

Understanding service discovery in Kubernetes and its interaction with a service mesh like Istio requires knowledge of both native Kubernetes mechanisms and the additional capabilities provided by Istio.

Native Kubernetes Service Discovery

Kubernetes natively supports service discovery using the following concepts:

1. Pods: The smallest deployable units in Kubernetes, which can run one or more containers.
2. Labels and Selectors: Pods are labeled with key-value pairs. Services use selectors to select a group of pods based on their labels.
3. Services: Kubernetes Services are abstractions that define a logical set of Pods and a policy to access them. A Service has an IP address and port that, unlike individual Pods, remain constant.
4. DNS for Service Discovery: Kubernetes runs a DNS pod and service for cluster-internal DNS. When a Service is created, it gets a DNS entry. This allows Pods to perform service discovery automatically using the service’s DNS name.

Example: ServiceA Communicating with ServiceB

• ServiceA sends a request to ServiceB using the DNS name, like serviceb.default.svc.cluster.local.
• The DNS name resolves to the cluster IP of ServiceB.
• The request is routed to one of the Pods selected by ServiceB based on load-balancing.

Service Discovery with Istio

When Istio is implemented in a Kubernetes cluster, it enhances the existing service discovery mechanism:

1. Istiod (Control Plane): Acts as the central component managing the configuration and state of the service mesh.
2. Envoy Sidecars: Each pod gets an Envoy sidecar proxy that intercepts all inbound and outbound traffic.

Example: ServiceA Communicating with ServiceB via Istio

• ServiceA sends a request to ServiceB.
• Instead of going directly to ServiceB, the request is intercepted by ServiceA’s Envoy sidecar.
• The sidecar proxy queries Istiod for the location and health of ServiceB‘s endpoints.
• Istiod responds with the appropriate endpoints of ServiceB.
• The sidecar then forwards the request to one of these endpoints, typically another sidecar proxy in front of ServiceB‘s Pod.

What Happens If ServiceB’s Pods Disappear?

• Kubernetes Behavior: The endpoints of ServiceB are updated, and any requests to ServiceB would fail to find a valid endpoint.
• Istio’s Response:
• The Envoy sidecars are constantly informed by Istiod about the state of the service mesh, including the availability of service endpoints.
• If ServiceB‘s Pods disappear, Istiod updates the sidecars with this information.
• ServiceA’s sidecar, upon not finding healthy endpoints for ServiceB, can apply resilience features like retries, circuit breaking, or failing over to another service as per the configured rules.
• This dynamic update and advanced traffic management capabilities of Istio help in maintaining the stability and reliability of the service communication.

Comunications Between isitod and Envoy Sidecars

Istiod communicates with the Envoy sidecars in an Istio service mesh using a combination of event-driven updates and periodic polling, depending on the situation. The process relies on the xDS (eXtensible Discovery Service) API, which Envoy uses to dynamically discover network resources. Here’s how it works:

xDS API in Envoy

• xDS API: Envoy uses the xDS API for dynamic resource discovery. This API includes various discovery services like LDS (Listener Discovery Service), CDS (Cluster Discovery Service), EDS (Endpoint Discovery Service), and RDS (Route Discovery Service).

Istiod communicates with Envoy sidecars using the xDS API over a gRPC stream. This communication is primarily event-driven, reacting to changes in the service mesh, but also includes periodic polling to ensure configuration consistency.

Istio as a Service Discovery Component in Kubernetes

Kubernetes provides a robust native service discovery mechanism based on DNS and Services. Istio enhances this by introducing a service mesh layer that enables advanced traffic management, resilience features, and real-time updates on service availability, providing a more dynamic, secure, and efficient communication process between services like ServiceA and ServiceB.

1. Integration with Kubernetes: In a Kubernetes environment, Istio integrates seamlessly, extending its functionalities. Kubernetes itself provides service discovery and load balancing for services running on it. Istio complements these features.
2. Service Discovery: Istio uses the services registered with Kubernetes for service discovery. When a service in Kubernetes is created, Istio will automatically detect it and use it in its routing rules. This is crucial in a microservices architecture where knowing the location of a service is essential for communication.
3. Load Balancing and Traffic Routing: Through its intelligent routing capabilities, Istio can control the flow of traffic and API calls between services. This includes advanced load balancing across multiple instances of a service.
4. Dynamic Updates: As services are scaled up or down in Kubernetes, Istio dynamically adjusts its routing rules. This means that the service discovery component is always up-to-date with the current state of the Kubernetes cluster.
5. Fault Injection and Traffic Shaping: Istio can also simulate faults or changes in network behavior, which helps in testing the resilience of the system. It can also shape the traffic based on various criteria, aiding in blue/green deployments, A/B testing, etc.

Istio automatically detects new services in a Kubernetes (K8s) environment through its integration with Kubernetes’ service discovery mechanisms.

Kubernetes Service Discovery Basics

1. Kubernetes Services: In Kubernetes, a ‘Service’ is an abstraction which defines a logical set of Pods (the smallest deployable units in Kubernetes) and a policy by which to access them. Services in Kubernetes serve as a key part of its internal service discovery mechanism.
2. Endpoints: Kubernetes maintains a list of ‘Endpoints’, which are IP addresses and ports where Pods can be reached. These Endpoints are updated whenever Pods are added, removed, or changed.
3. Labels and Selectors: Services are linked to Pods through labels and selectors. A service’s selector matches labels on Pods to include them in its set.

Istio’s Integration with Kubernetes Service Discovery

1. Watching the Kubernetes API Server: Istio’s control plane components, specifically the Pilot (or Istiod in newer versions), keep a watch on the Kubernetes API server. They listen for changes in services, pods, and endpoints.
2. Automatic Detection: When a new service is created in Kubernetes, it’s registered with the Kubernetes API server. Istio’s control plane detects this new service by watching the API server.
3. Service and Endpoint Information: Istio retrieves information about the service and its endpoints (i.e., the IP addresses and ports of the Pods that form the service) from Kubernetes.
4. Updating Envoy Proxies: After detecting a new service and its associated endpoints, Istio’s control plane updates the Envoy proxies (which run as sidecars to each service’s Pod). It provides them with the latest routing information, ensuring that traffic can be correctly routed to the new service.
5. Dynamic Configuration: This process is dynamic and happens automatically as services are added, updated, or removed in Kubernetes. Istio ensures that the Envoy proxies always have up-to-date information to properly route requests.
6. Service Mesh Configuration: In addition to basic service discovery, Istio allows you to define more advanced traffic management rules, such as canary deployments, A/B testing, or circuit breakers, and applies these configurations dynamically as well.

Istio provides a powerful and dynamic environment for managing microservices communication in a Kubernetes cluster. This seamless integration is one of the reasons why Istio is a popular choice for service mesh implementation in Kubernetes environments.

When using Istio, or specifically the Envoy sidecars that Istio deploys with each service in a Kubernetes (K8s) environment, the communication between Service A and Service B is managed and facilitated by these sidecars. Here’s an overview of how this communication process works and how Service A knows about the connectivity and the nodes of Service B:

Basic Communication Process

1. Deployment of Envoy Sidecars: In an Istio-enabled K8s cluster, each pod typically has an Envoy proxy sidecar. This sidecar intercepts all incoming and outgoing network traffic to the pod.
2. Service A Initiates Communication: When Service A wants to communicate with Service B, it sends a request. Instead of sending this request directly to Service B, it sends it to its own Envoy sidecar.
3. Service Discovery by Istio: The sidecar uses the service discovery information provided by Istio, which is aware of all services in the K8s cluster, their endpoints, and their current state.
4. Routing Decision: The Envoy sidecar of Service A uses Istio’s dynamic service discovery and routing rules to determine where to send the request. It selects the appropriate instance of Service B based on the load balancing and routing configuration.
5. Forwarding the Request: The request is then forwarded to the chosen instance of Service B, specifically to the Envoy sidecar of that instance.
6. Reaching Service B: The Envoy sidecar of Service B then passes the request to the actual Service B application.

Service Availability and Load Information

1. Health Checks: Envoy proxies can be configured to perform health checks on services. These checks determine whether an instance of a service is healthy and can handle requests.
2. Load Balancing: Envoy sidecars use various algorithms for load balancing. They are aware of each instance of Service B and their health status. This information is used to intelligently distribute the load.
3. Dynamic Updates: Istio’s control plane (Istiod) monitors the K8s API for changes in services, pods, and their health statuses. It updates the Envoy proxies with this information, ensuring they have the latest data about the availability and health of Service B.
4. Circuit Breaking: Istio can be configured to implement circuit breaking patterns. If Service B becomes overloaded or starts failing, the circuit breaker can trip, preventing further requests to Service B, ensuring system stability.
5. Resilience Features: Features like retries, timeouts, and failovers are also managed by Istio, providing additional mechanisms to ensure that communication is resilient and reliable.