diff --git a/custom-per-route-options.html.md.erb b/custom-per-route-options.html.md.erb index 706878b0..7c715472 100644 --- a/custom-per-route-options.html.md.erb +++ b/custom-per-route-options.html.md.erb @@ -8,10 +8,12 @@ By default, communication between Gorouter and backends is configured through th This topic describes how to specify per-route Gorouter options scoped at the application level. This greater granularity lets developers tailor optimal routing behavior for applications' unique load profiles or other requirements. -Gorouter supports the following per-route option, described in the section below: +Gorouter supports the following per-route options, described in the sections below: - `loadbalancing`: Configures the load balancing algorithm used by Gorouter for this particular route. <%= vars.per_route_lb_version %> - - Settings: `round-robin`, `least-connection`. + - Settings: `round-robin`, `least-connection`, `hash`. +- `hash_header`: Defines the header Gorouter uses for routing decisions on this route. Required when `loadbalancing` is set to `hash`. Cannot be used with other load balancing algorithms. <%= vars.hash_routing_version %> +- `hash_balance`: Sets the float number for the balance factor used by Gorouter to manage load imbalance applying the hash-based routing for this route. Optional when `loadbalancing` is `hash`. Cannot be used with other algorithms. <%= vars.hash_routing_version %> ## Configure Gorouter's Load Balancing Algorithm @@ -19,13 +21,13 @@ Gorouter supports the following per-route option, described in the section below The per-route option `loadbalancing` allows configuring the load balancing algorithm, which defines how the load is distributed between Gorouters and backends. -This option supports two settings for load balancing: +This option supports three settings for load balancing: - `round-robin` distributes the load evenly across all available backends - `least-connection` directs traffic to the backend with the fewest active connections at any given time, optimizing resource utilization +- `hash` distributes requests based on a hash of a specific HTTP header, ensuring requests with the same header are consistently directed to the same backend. <%= vars.hash_routing_version %> - -### Configure Load Balancing in an App Manifest +### Configure Load Balancing using an App Manifest To configure per-route load balancing for an application that has not yet been pushed: @@ -49,91 +51,155 @@ To configure per-route load balancing for an application that has not yet been p cf push -f manifest.yml ``` -### Change Load Balancing Algorithm of an Existing Route +### Create a Route with a Specific Load Balancing Algorithm Using the CF CLI + +To create a route with a per-route `loadbalancing` option, you can use the CLI command `create-route`. +For example: + +```console +cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin +``` + +### Map a Route to an Existing App with a Specific Load Balancing Algorithm Using the CF CLI + +To create and map a new route to an existing application with the per-route `loadbalancing` option, you can use the CLI command `map-route`. + +For example: + +```console +cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin +``` + +

+The command map-route supports the --option flag only for new routes. +To update an existing route, use the command update-route described below.

+ + +### Update the Load Balancing Algorithm of an Existing Route Using the CF CLI -To change the per-route `loadbalancing` option of an existing route, you can use the cli command, `update-route`. +To change the per-route `loadbalancing` option of an existing route, you can use the CLI command `update-route`. For example, to change an app route's algorithm from `least-connection` to `round-robin`, you can run the `update-route` command: ```console cf update-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin ``` +### Remove the Specific Load Balancing Algorithm Using the CF CLI -Alternatively, it is also possible to update the per-route load balancing option via the `/v3/routes` API. - -Run the `PATCH` request to the targeted API endpoint: +To remove the `loadbalancing` option from an existing route, run: ```console -cf curl /v3/routes/GUID -X PATCH -H "Content-type: application/json" \ - -d '{ - "options": { - "loadbalancing": "round-robin" - } - }' +cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing ``` -Where `GUID` is the unique identifier for the route. -### Create a Route with a specific Load Balancing Algorithm +## Configure Hash-Based Routing -To create a route with a per-route `loadbalancing` option, you can use the cli command `create-route`. -For example: +<%= vars.hash_routing_version %> + +Hash-Based Routing is a load-balancing method that routes incoming requests to application instances using a hash of a specific HTTP header value. This ensures consistent routing, so requests with the same header value always go to the same instance. The per-route hash options offer detailed control over hash-based load balancing for individual routes. + +### Configure Hash-Based Routing with an App Manifest +1. In the application manifest, include a `route` definition with the following `options` attributes: + - `loadbalancing` set to `hash` + - `hash_header` set to the HTTP header name used for routing decisions + - optionally, `hash_balance` set to a float number for the balance factor used by Gorouter to [manage load imbalance](hash-based-routing.html#handling-imbalance-loads) for this particular route. + + ```yaml + --- + applications: + - name: MY-APP + routes: + - route: MY-HOST.EXAMPLE.COM + options: + loadbalancing: hash + hash_header: HASH-HEADER-NAME + hash_balance: 1.2 + ``` + + Where `MY-APP` is the name of your app, `MY-HOST.EXAMPLE.COM` is the route you want to map to your app and `HASH-HEADER-NAME` is the HTTP header name. + +1. Push the app with the manifest: + + ```console + cf push -f manifest.yml + ``` + +### Create a Route with Hash-Based Options Using the CF CLI + +To create a route with hash-specific options, you can use the CLI command `create-route`. For example: ```console -cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin +cf create-route EXAMPLE.COM --hostname MY-HOST --option loadbalancing=hash --option hash_header=HASH-HEADER-NAME --option hash_balance=1.2 ``` -Alternatively, it is also possible to create a route with a per-route load balancing option via the `/v3/routes` API: +Alternatively, you can use the shorthand `-o` for `--option`. Since `hash_balance` is optional, you can omit it: ```console -cf curl /v3/routes -X POST -H "Content-type: application/json" \ - -d '{ - "host": "MY-HOST", - "path": "MY-PATH", - ... - "options": { - "loadbalancing": "round-robin" - } - }' +cf create-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME ``` -### Map a Route to an Existing App with specific Load Balancing Algorithm +### Map a Route with Hash Options to an Existing App Using the CF CLI -To create and map a new route to an existing application with the per-route `loadbalancing` option, you can use the cli command `map-route`. +To create a new route suitable for hash-based routing and map it to an existing application, you can use the CLI command `map-route`. For example: ```console -cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST --option loadbalancing=round-robin +cf map-route MY-APP EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME -o hash_balance=1.2 ```

The command map-route supports the --option flag only for new routes. -To update an existing route, the command update-route must be used as described before.

+To update an existing route, see the instructions for update-route below. -### Retrieve Route Options +### Update an Existing Route with Hash Options Using the CF CLI -To read route options, you can query the route using the `route` command: +You can change an existing route that uses the default load balancing algorithm to the hash load balancing algorithm. + +For example, to change an app route's algorithm from default `round-robin` to `hash` and set `hash_header` to HASH-HEADER-NAME without a balance factor, you can run the `update-route` command: ```console -cf route EXAMPLE.COM --hostname MY-HOST +cf update-route EXAMPLE.COM --hostname MY-HOST -o loadbalancing=hash -o hash_header=HASH-HEADER-NAME ``` - The response lists the chosen `loadbalancing` algorithm option, e.g. `least-connection`: +To add a balance factor along with the previous settings, you can later run the `update-route` command, for example, with the `hash_balance` option set to `1.5`: ```console -options: {loadbalancing=least-connection} +cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=1.5 ``` - Alternatively, you can query the `routes` API endpoint for a route: +To unset the balance factor, run the `update-route` command with `hash_balance` set to 0. ```console -cf curl /v3/routes/?hosts=MY-HOST +cf update-route EXAMPLE.COM --hostname MY-HOST -o hash_balance=0 ``` -Where `MY-HOST` is the host attribute of the route. The response lists the chosen `loadbalancing` algorithm option as well: +Setting the balance factor to 0 indicates to Gorouter that load imbalance is accepted and all requests for a particular hash should be routed to the same instance as long as it's healthy, without redirecting to other predetermined instances. + +### Revert Hash Options Using the CF CLI + +Running the `update-route` command with the `-r` flag for the option `loadbalancing` removes all hash options from the route, returning to the default load balancing algorithm: + +```console +cf update-route EXAMPLE.COM --hostname MY-HOST -r loadbalancing +``` + +## Retrieve Route Options + +To view route options, you can query the route using the `route` command: ```console - "options": {"loadbalancing": "least-connection"} +cf route EXAMPLE.COM --hostname MY-HOST ``` -To retrieve all the routes with the corresponding options in a space of an organization, you can use the `routes` command. +The response lists the chosen `loadbalancing` algorithm option, e.g. `least-connection`: + +```console +options: {loadbalancing=least-connection} +``` + +Or `hash` with its related options: + +```console +options: {hash_balance=1.2, hash_header=HASH-HEADER-NAME, loadbalancing=hash} +``` diff --git a/deploy-apps/manifest-attributes.html.md.erb b/deploy-apps/manifest-attributes.html.md.erb index 3ca57e61..addf087d 100644 --- a/deploy-apps/manifest-attributes.html.md.erb +++ b/deploy-apps/manifest-attributes.html.md.erb @@ -575,7 +575,9 @@ Under each route, you can optionally include an `options` attribute to configure Available options are: -- `loadbalancing` - defines how Gorouter distributes requests across the application backends. Valid values are `round-robin` and `least-connection`. +- `loadbalancing` - defines how Gorouter distributes requests across the application backends. Valid values are `round-robin`, `least-connection` and `hash`. +- `hash_header` - defines the header used for hash-based routing decisions. Required when `loadbalancing` is set to `hash`. +- `hash_balance` - defines the balance factor used to manage load imbalance for hash-based routing. Optional when `loadbalancing` is set to `hash`. Values in the 1.1-2.0 range provide the best balance of even distribution and performance. Omitting `hash_balance` or setting it explicitly to 0 indicates that the load situation will not be considered. For example: @@ -589,6 +591,11 @@ For example: - route: example2.com options: loadbalancing: least-connection + - route: example3.com + options: + loadbalancing: hash + hash_header: Hash-Relevant-Header + hash_balance: 1.25 ``` #### Manifest attributes diff --git a/hash-based-routing.html.md.erb b/hash-based-routing.html.md.erb new file mode 100644 index 00000000..930f534a --- /dev/null +++ b/hash-based-routing.html.md.erb @@ -0,0 +1,60 @@ +--- +title: Hash-Based Routing +owner: CF for VMs Networking +--- + +## Purpose + +Hash-Based Routing is a load-balancing algorithm that distributes incoming requests to application instances based on a hash of a specific HTTP header value, e.g., `X-Resource-ID` or `Tenant-ID`. This ensures consistent routing behavior where requests containing the same header value are always directed to the same instance. + +## Prerequisites + +To use Hash-Based Routing, ensure that + +- your <%= vars.app_runtime_abbr %> deployment is running version <%= vars.hash_routing_version %> or later. +- platform operators activate the feature so that the CF feature flag `hash_based_routing` is set to `true`. See [Feature Flags](https://docs.cloudfoundry.org/adminguide/listing-feature-flags.html#flags) for more details. + +## Key Features + +- **Configurable via Per-Route Options**: Hash-Based load-balancing setup through application route options +- **Configurable Hash Header**: The HTTP header to consider for Hash-Based Routing is configurable for each route +- **Session Affinity Precedence**: Session affinity (sticky sessions) is prioritized over Hash-Based Routing +- **No availability zones preference**: The global properties `locallyOptimistic` and `localAvailabilityZone` are ignored when using Hash-Based Routing +- **Consistent Hashing**: Implementation of the Maglev Algorithm (see [Maglev: A Fast and Reliable Software Network Load Balancer](https://storage.googleapis.com/gweb-research2023-media/pubtools/2904.pdf) for details) +- **Minimal Rehashing**: Usage of the Maglev lookup table to map application instances by hash ensures that hash positions are shifted to other instances as little as possible when application instances are added or removed +- **Handling imbalanced loads**: Detection and mitigation of higher request loads on individual instances prevents overloading while keeping instances for a particular hash at a minimum. + +Hash-Based Routing implements a clear precedence hierarchy: + +1. **Sticky Sessions**: First checks if a sticky session is used and the sticky session endpoint is available +2. **Hash-Based Routing**: Calculates the hash of the specified HTTP header value and routes the requests to a pre-determined application instance +3. **Default Load Balancing**: Falls back to default load balancing if the header configured in the application route for Hash-Based Routing is absent in the request + + +## Hash-Based Routing vs. Session Affinity + +Using Session Affinity, requests from a particular client are always routed to the same app instance. This allows apps to store session data specific to a user session (see more details in the [Session Affinity](https://docs.cloudfoundry.org/concepts/http-routing.html#-session-affinity) documentation). + +While Session Affinity offers a solution for maintaining session consistency, it poses scalability challenges, as Session Affinity is confined to a single instance. It increases the risk that large customers could be routed to the same instance by chance. On the other hand, enabling Session Affinity requires additional implementation effort on the application side to return a sticky session cookie in responses. + +In contrast, Hash-Based Routing provides a more scalable and balanced approach by consistently distributing requests based on a hash of a specific HTTP header value. This hash value determines the appropriate application instance for each request, ensuring that requests with the same hash are consistently routed to the same instance, but might be routed to another predetermined instance when the current one is saturated (find more about Handling imbalanced loads below). In contrast to Session Affinity, no code change is required. + +This makes Hash-Based Routing particularly suitable for applications that demand high scalability and performance, such as microservices architectures or multi-tenant applications, especially when dealing with limited or memory-intensive backend resources. + +## Handling imbalanced loads + +Hash-Based Routing includes mechanisms to detect imbalanced load across application instances. An imbalanced load occurs when certain hashes receive more traffic, such as when a specific tenant generates many requests, resulting in heavier use of their mapped instances than others’. Additionally, multiple high-traffic targets might be assigned to the same instance. + +To prevent overloading specific instances while others remain underutilized, the acceptable threshold for load imbalance can be configured using the `hash_balance` property. +This factor determines whether an instance is handling more traffic than its fair share based on the average load across all instances, measured by the number of in-flight requests. For example, with a balance factor of 1.25, no single instance should handle more than 125% of the average number of in-flight requests across all instances managed by the current router. When this threshold is exceeded, the router redirects subsequent requests to other, less-loaded instances. +Values of the balance factor in the 1.1-2.0 range offer a good balance between even distribution and performance. However, optimal values depend on the application's traffic patterns and load characteristics. + +This approach ensures that a minimum number of instances process requests for a particular hash while preventing any single instance from becoming overloaded. + +## Minimal Rehashing + +The Maglev algorithm used in Hash-Based Routing minimizes rehashing when application instances are added or removed. When a new instance is added, only a small subset of hashes is remapped to it, while most continue to route to their original instances. Similarly, when an instance is removed, only the hashes mapped to that instance are reassigned. This design minimizes disruption and maintains consistent routing behavior as the application scales up or down. + +## Retries in Hash-Based Routing + +For idempotent requests, Hash-Based Routing supports a retry mechanism. If a request fails due to a network error or a 5xx response from the application instance, the router retries the request with a different, predetermined application instance. The next entry in the Maglev lookup table determines this instance. This approach aligns with the approach to handling imbalances. It ensures that the retry mechanism adhere to the principles of Hash-Based Routing while providing resilience against transient failures.