How JupiterOne's DevOps Team Uses CAASM to Understand Service Coupling & Deployment Order

By

In the current microservice landscape, there are more than enough automation tools and infrastructure as code (IaC) solutions to make deployments swift and effortless. This is great for managing and deploying a single service, but what happens when there's a need to deploy your entire product to a new environment? Specifically, how do you know which services need to be deployed and in what order?

In this post, I'll discuss how JupiterOne's DevOps team leverages JupiterOne to understand service coupling and deployment order, by mapping CodeRepo USES CodeRepo relationships. We also use an Insights dashboard to help answer questions about deployment order.

What is a Deploy Dependency?

A deploy dependency is a service whose outputs are used as inputs in another service (specifically Terraform outputs). Based on that, we know if service B must be deployed prior to service A. Ideally, we would like to see a relationship in JupiterOne between these two CodeRepo entities.

Where are Deploy Dependencies Defined?

We define these deploy dependencies in a dependencies.yaml file that exists in all of our deployable repositories.

Example dependencies.yaml for an account-service:

---
terraform:
- user-service
- invitation-service
- feature-toggle-service

Since the account-service USES the outputs of the services user-service, invitation-service, and the feature-toggle-service, we know that those must be deployed prior to the account-service itself.

By answering the questions above, we have what we need to automate the creation of CodeRepo USES CodeRepo relationships in JupiterOne.

Our team developed a script that parses the dependencies.yaml of our deployable repositories in GitHub, creates relationships based on the matching CodeRepo entities with the following properties:

const relationshipKey = `${mainRepo.entity._key}|uses|${dep.entity._key}`;
const relationship: RelationshipForSync = {
_key: relationshipKey,
_class: 'USES',
_type: 'repo_dependency',
_fromEntityId: mainRepo.entity._id,
_toEntityId: dep.entity._id,
deployRelationship: true,
};
payload.push(relationship);

 

Next, we do a bulk upload into JupiterOne using the Node.js Client and CLI, which is available on our GitHub.

const jobState = await J1Client.bulkUpload ({
scope: 'summary-relationships-code-repo-dependencies-workload',
relationships: payload,
});

Currently, our DevOps team runs this on a daily cadence to keep our deploy dependency relationships up to date.

The JupiterOner node.js API client wrapper and CLI utility supports uploading entities in either JSON or YAML format. For another example use case, we recommend: Using JupiterOne as a central repository for SecOps and compliance artifacts.

How to Visualize Deploy Dependencies

Now that the relationships exist in JupiterOne we can start to visualize our deploy dependencies. Here is an example JupiterOne query that shows us the deploy dependencies of the account-service example used above:

Find CodeRepo with name = 'account-service' that uses >> CodeRepo return TREE

01-visualize-deploy-dependencies

 

We also created a useful dashboard to get an overview of our deployment coupling using JupiterOne Insights.

02-insights-dashboard-deploy-dependencies

 

You can learn how to get started with prebuilt or customized Insights Dashboards here.

 

Repository Deployment Coupling

Here is the configuration we use for the first widget displayed on our DevOps team's Insights dashboard above, which creates the pie chart labeled "Repository Deployment Coupling":

Find CodeRepo with tag.AccountName="YourAccount" as f
that USES as rx CodeRepo
where rx.deployRelationship = true
return f.displayName as none, count(rx) as value

03-repository-deployment-coupling

 

Repository Deploy Dependency Direction

And here are the configurations for the second widget, the lower bar chart labeled "Repository Deploy Dependency Direction":

Find CodeRepo with tag.AccountName='YourAccount' as f
that USES << as rxIn CodeRepo
return f.displayName as x, count(rxIn) as y
order by y DESC

04-repository-deploy-dependency-direction

 

Find CodeRepo with tag.AccountName='YourAccount' as f
that USES >> as rxOut CodeRepo
Return f.displayName as x, count(rxOut) as y
Order by y ASC

05-repository-deploy-dependency-direction

 

Future Goals: Identify Direct and Indirect Circular Dependencies

These relationships have proven to be extremely useful in determining deployment order, but there is quite a bit of extra insight we could get by iterating on these relationships. For example, we could start to identify circular dependencies (direct and indirect) and with additional logic determine a specific order for services to be deployed. A shared dashboard for common understanding of deploy dependencies has been a valuable tool for our DevOps team, so we look forward to visualizing these additional DevOps and SRE use cases.

For more use cases and insight into how JupiterOne's team uses our own solution, we recommend the JupiterOne SecOps Automation Examples repo on Github.

Neal Patel
Neal Patel

Neal Patel is a DevOps Engineer who focuses on improving CI/CD, developer tooling, and observability. When, he's not on the clock you can find him getting cutoff by bartenders for pouring shots for his friends :zany_face:

To hear more from Neal, get our newsletter. No spam, just the good stuff once or twice a month. Sign up below.

Keep Reading

What’s new in JupiterOne: Reducing time to value with the new Query Builder (Part 2)
February 6, 2023
Blog
What’s new in JupiterOne: Reducing time to value with the new Query Builder (Part 2)

The new JupiterOne Query Builder streamlines your querying experience by eliminating errors, simplifying query builds, and reducing time to value.

The top 10 questions that every engineering leader should be able to answer
February 2, 2023
Blog
The top 10 questions that every engineering leader should be able to answer

We polled some of our engineering leaders to see what it takes to succeed. In part two, we see if their answers align with the CISOs we talked to.

Identify compromised versions of Github using JupiterOne
January 31, 2023
Blog
Identify compromised versions of GitHub apps using JupiterOne

As a preventative measure, Github will be deprecating the Mac and Windows signing certificates used to sign Desktop app versions 3.0.2-3.1.2 and Atom versions 1.63.0-

15 Mar 2022
Blog
One line headline, one line headline

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud eiut.

15 Mar 2022
Blog
One line headline, one line headline

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud eiut.

15 Mar 2022
Blog
One line headline, one line headline

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud eiut.