Saving each event performed with Clutch for security auditing is a top-level concern, and the processes to accomplish this fit into the normal component architecture.
Clutch has configurable middleware to save data on each incoming request, a module to query for previous events, and services to store and forward events to new sinks. What information it stores for the request comes from API annotations on the request itself. The below table has the components ordered as they are in a call-chain.
|Module to retrieve audit events, called from outside of Clutch and calls the audit service.|
|Middleware that runs on each request. Calls the audit service to store request information.|
|Handler in front of storage services. Called to save or retrieve request information.|
|Calls the audit service to retrieve request information. Call out of Clutch to forward the events.|
The annotations are read by middleware for each request and response, which forwards the information to the audit service for storage. The audit service is responsible for persistence as well as forwarding to other sinks to fan-out the messages. Each step of saving and forwarding is configurable with filters based on event attributes.
Clutch's auditing middleware reads the value of two API annotations: the request "type" to determine what sort of action is being performed on a resource, and the resource name to identify what item is being modified.
These annotations let customizable data on each request be logged and reviewed.
Clutch will save the the "type" of action that was performed (create, read, update, or delete or CRUD operations) for each request. The request type comes from an API annotation on each RPC.
AmiiboAPI example proto, to show that the
GetCharacters endpoint is a "read" operation, an extra
option annotation would be needed.
The annotation would look similar for the other supported types of changes.
Clutch will also resolve what resources were modified by each request to their unique identifiers. This is also accomplished through API annotations.
There are two different annotations used to mark the resources in a single request:
fieldsannotation will indicate what fields contain resource names.
patternsannotation will try template substitution with field values in order to create a name.
If we wanted to add these annotations to the Amiibo example, where the identifier for a character is a combination of its name and game series, we would make the following modifications to the API definition.
The first annotation (
fields = "characters") says that to get the set of resources contained in this message, to look at the
The second annotation (
patterns = ...) says that the identifier for this message is built by combining the value of the
name field with the value of the
On each request, the above annotations are read along with what endpoint was called, by whom, and when. When a response is being sent, any additional resources or final identifiers are merged with the initial list from the request.
All of this information is passed along to the audit service to persist.
The audit service has two behaviors: write requests somewhere, and read them back out. It takes events from the middleware and saves them, and it also pushes them to later "sinks" for further processing.
Clutch ships with an implementation that uses Postgres as backing storage. With this, events that are saved can be filtered. At Lyft, we use this to prevent saving healthcheck information, since it is not interesting to us from a security perspective.
The configuration for this looks like:
Sinks asynchronously propagate events to other systems after they are persisted to Clutch's database.
Clutch ships with a logging sink as a scaffold for your own, as well as a sink for Slack.
Adding and customizing audit sinks lets you save or process infrastructure events however appropriate for your needs.
Clutch's audit events can also be viewed by querying the audit module if it is enabled.
Local Adhoc Use
Clutch ships with an in-memory storage for events which allows it to be used without setting up a Postgres database. It is not recommended to run this way outside of a trial or adhoc temporary use. All history of actions taken with Clutch will be lost with the process shutting down.
Recommended Production Setup
Below is sample configuration to show how the services described are enabled. Note that because services are instantiated in the order they are listed, order matters! Since the audit service depends on both the database and the sink, it needs to be listed after them.