Topology API's

The Topology feature set powers core Clutch capabilities such as autocomplete, as well as providing API's that can be leveraged for a multitude of purposes.

One of the main goals of the topology service was to create an extensable caching mechanism, to store all aspects of infrastructure with the ability for it to be easily accessed via APIs. This allows implementers to write features that are not bound by service provider API rate limits and latencies and allows for Clutch to provide a consistent user experience that scales with your users.

Topology Caching

At the core of the topology service is its caching functionality which is the foundation that powers its APIs.

You can enable the cache by adding the following to your clutch configuration.

clutch-config.yaml

services:
  ...
  # The topology services requires the postgres datastore to be configured
  - name: clutch.service.topology
    typed_config:
      "@type": types.google.com/clutch.config.service.topology.v1.Config
      cache: {}

There are additional configuration options that can be tuned if necessary.

Leader Election

Clutch elects a leader to handle the caching operations to ensure that only one instance performs write heavy operations.

How to extend the Topology Cache

Extending the topology cache for private gateways can be done by implementing the CacheableTopology interface.

type CacheableTopology interface {
  CacheEnabled() bool
  StartTopologyCaching(ctx context.Context, ttl time.Duration) (<-chan *topologyv1.UpdateCacheRequest, error)
}

If implemented, the topology service will begin caching resources provided by the services which implement the interface, storing them in the topology_cache table to utilize for features such as the Search API which powers autocomplete.

An example of a the Topology interface being implemented can be found in he Kubernetes service caching implementation.

Config Load Order

It's important to load the clutch.service.topology last or near the bottom in your Clutch configuration. This is required since the topology service will iterate through the service registry at load time to see if any services implement the CacheableTopology interface to determine if caching should be enabled.

Autocomplete

Once the topology service and module are configured autocomplete for all known Clutch resources and workflows will be enabled by default.

clutch-config.yaml

modules:
 - name: clutch.module.topology
services:
  ...
  - name: clutch.service.k8s
    typed_config:
      "@type": types.google.com/clutch.config.service.k8s.v1.Config

  # The topology services does require the postgres datastore to be configured
  - name: clutch.service.topology
    typed_config:
      "@type": types.google.com/clutch.config.service.topology.v1.Config
      cache: {}

Autocomplete for custom resolver types

If you have custom types within your gateway you can enable autocomplete by following the steps below.

1. Enable the searchable annotation on the resolver proto, the example below is for Kubernetes pods.

message PodID {
  option (clutch.resolver.v1.schema) = {
    display_name : "pod ID"
    search : {enabled : true}
  };
  ...
}

2. You will likely have a custom resolver for custom types to satisfy the autocomplete function that is present on the resolver interface. There are many examples throughout Clutch you can reference including the Kubernetes resolver implementation. You will notice that the autocomplete implementation for the resolver is almost identical to the AWS and Core Clutch resolver implementations.

3. Enabling autocomplete on the frontend varies depending on the implementation of the workflow. if you are using the resolver component, autocomplete will be enabled without any additional effort. However if you have a more custom workflow or page, you enable autocomplete on any TextField by adding the following.

const autoComplete = async (search: string): Promise<any> => {
  // Check the length of the search query as the user might empty out the search
  // which will still trigger the on change handler
  if (search.length === 0) {
    return { results: [] };
  }

  const response = await client.post("/v1/resolver/autocomplete", {
    // Replace with the type you want to autocomplete on
    want: `type.googleapis.com/clutch.core.project.v1.Project`,
    search,
  });

  return { results: response?.data?.results || [] };
};


<TextField
  name="sometextfield"
  // Specify the autocompleteCallback prop for the TextField component.
  autocompleteCallback={v => autoComplete(v)}
/>