Fingerprinting Knock errors for Sentry in Elixir



At SwayDM, we use the fantastic Knock platform to define notification workflows, and deliver emails, and SMSs to our users.

Our codebase is written in Elixir, and we use Sentry for error reporting via sentry-elixir.  One thing you can do with sentry-elixir is report an event to Sentry every time your app logs at the error level.  You can also specify which metadata logger keys are forwarded to sentry as metadata attributes.

This works great in conjunction with Knock, but if you only log the error, all your Knock related issues in Sentry will be grouped as one, regardless of which identify, or workflow caused them.   This is where custom fingerprinting comes in.

Configure Sentry

First, configure sentry to define a module, and function to be invoked before the event is sent to Sentry.

You must have the Sentry LoggerBackend configured.  If you want to pass logger metadata to sentry, you must explicitly add it to this configuration.



Capture Logger Metadata

Capture details in the logger metadata before logging the error.  These functions are defined in a module that implements a Behavior named SwayDM.Notifications.Provider so we can easily stub or mock out notification calls.

We currently only make calls to two different knock resources.  If we were calling more, we could probably make a nice wrapper, or more generic function that could handle and log all knock responses.

Calls to Knock.Users.identify


Calls to Knock.Workflows.trigger




Create before_send Function

Next, create a before_send function in your new Sentry module.   We use pattern matching on function heads to determine which before_send function is invoked.   

The before_send function takes in a sentry event, and returns a sentry event.  You can make any modifications you see fit to the event.  This is great for when you want to grab more info out of an error, or log message, especially if it is supplied via logger metadata or buried in an error message. 

It is also a good method to redact sensitive information that may be accidentally sent to Sentry. I've seen cases where a match error gets inspected, and details of the matched data structures gets sent to sentry. This can compromise PII, or other secrets if care is not taken.

Creating the Fingerprint

In this case, we are supplying a custom event message, and fingerprint.  The fingerprint is just a list of strings.  The order needs to be consistent, as it is hashed by sentry to group events efficiently.

We are setting an initial namespace of ["elixir_app", "knock"].  This is just a safeguard to know our fingerprint will not accidentally interfere with other events (very unlikely, but a good practice).  Next, we supply [workflow, code], which are pulled from the metadata, and knock error.   

Now, rather than all Knock errors being grouped together, they are grouped by the combination of workflow and code.  You could group by other attributes though if desired.


How events how show up in Sentry issue list:

How the event details look:

You can see our custom message, and the nicely formatted error.  The message was overwritten in our before_send function, and the nicely formatted logger_metadata values are due to the Logger.error invocation, and Sentry.LoggerBackend configuration.


Sentry Fingerprint:

Finally, you can see the event fingerprint in the Event Grouping Information on the Sentry event.



Sentry and Knock have saved me huge amounts of development, and debugging time.  I highly recommend both platforms.  If you have any questions, just ask!

- Andy Glassman


Comments

Popular posts from this blog

Write Admin Tools From Day One

Phoenix LiveView: Async Assign Pattern

Extremely Fake Cheapskates! Season One Recap