Safe On-Call Access to Prod with Sym and PagerDuty

Jon Bass
January 20, 2023

Sym is now listed in the PagerDuty Integration Directory! When I demo Sym, I often see the light bulb go on for prospects when I add in our PagerDuty integration. My demo flow starts with an end user who requests access with our Slack App. I then show how the requester needs a peer to review and approve their access. I point out that there’s already a ton of value in setting up peer reviewed approval flows. Then I go behind the curtain to look at our SDK, and that is where the PagerDuty magic comes in!

Peer reviewed access is an important security control, but it can add friction. Does everyone always need to wait for someone else to push a button to get their job done? What if it is 4 am and I’m responding to a page, do I need to wake someone else up too? In my demo, I respond to these concerns by wiring in PagerDuty. I show how with just a few lines of code, you can customize your flow to fast-track access for on-call users.

Here’s what a user with fast-tracked access might see when they make a request:

And here’s the code snippet that would produce this response:

When I use Sym’s SDK to integrate PagerDuty, prospects start to see why Sym is so well suited for platform developers. As my co-conspirator Adam wrote, we view platform teams as startups who need to think deeply about their internal customers. Sym’s SDK gives platform teams the tools they need to solve their access management pain points while still having empathy for the end users that need to adapt to new security controls.

Sym’s SDK includes native support for checking call status and other metadata from PagerDuty

“The ability to deliver effective abstractions might be the most defining skill of a great platform engineering team,” Adam argues. What we’re seeing at Sym is that an SDK-centric approach best enables these “effective abstractions”. Sym’s SDK lets teams rapidly get basic access flows going, but includes all the primitives that folks need to extend their flows to fit their department’s needs. Some SDK primitives like PagerDuty are fast to set up. Other primitives like HTTP or Lambda let you wire in custom stuff that we don’t (or can’t) know about, since it is specific to your department or organization.

How PagerDuty Helped Courier Streamline Their Access Flows

We’ve seen the value of our PagerDuty integration play out with many customers. A great example is Courier, one of Sym’s earliest design partners. Courier is an API and web studio for development teams to build and manage all product-triggered communications (email, chat, in-app, SMS, push, etc.) in one place.

Troy (CEO) and Seth (CTO) were committed to creating a culture where least privilege was the expectation for engineering and customer success even at an early stage. They used Sym to create access flows for both AWS and access to their own application. The Courier team used these flows to respond to requests from customers and handle sensitive infrastructure issues when necessary.

Courier’s access flows worked great… until they didn’t. “As our team and number of customers grew, we had to provide more support outside of normal working hours. Waiting for approvals became a bottleneck,” said Seth. That is when Seth updated some of their access flows to fast-track on-call access. In their updated setup, Courier still requires approval for certain escalated permissions, but they have removed their on-call bottleneck. They retain centralized logs and visibility of who has requested access, which is critical for their compliance program.

Seth not only saved his team valuable time by integrating PagerDuty with Sym, he also set Future Seth up for success by making his changes in code. The same change management controls that Courier uses for their application code now also support their access workflows. When it comes time for an audit, Seth can point to the pull request where he implemented their new on-call access policy. When the team decides to add some new resources or further modify their control logic, they can refer back to the code and our docs to safely iterate.

Sym Sets Teams Up for Platform Enablement

As Courier continues to grow, they’ll rely more and more on their internal developer platform to scale their engineering team. The components that are self-service and have great developer ergonomics will only become more important. This is where Sym’s “yes code” approach shines. Just like Sym helps teams distribute access decisions across the org, our SDK approach helps teams distribute the definition of the flows themselves across the org!  

Lets say, for example, that Courier expands its PagerDuty configuration to cover new schedules, incidents, policies, and more. Sym’s integration allows teams to pull in this metadata in new and creative ways. Different engineering departments can take the building blocks from the current flow implementations and tune these to meet their specific needs. The centralized platform team may get involved with review and log collection, but they don’t have to do all the work!

What Comes Next: More 'Aha' Moments

We’re so happy to have PagerDuty as a partner while we help platform teams enable dynamic and intelligent security processes throughout their organizations. I’m excited to share more PagerDuty “aha” moments with our customers as they discover how to wire these services together - to both move fast and to stay secure.

Recommended Posts