How JupiterOne's DevOps Team Uses CAASM to Understand Service Coupling & Deployment Order

by

In the current microservice landscape, there are more than enough automation tools and infrastructure as code (IaC) solutions to make deployments swift and effortless. This is great for managing and deploying a single service, but what happens when there's a need to deploy your entire product to a new environment? Specifically, how do you know which services need to be deployed and in what order?

In this post, I'll discuss how JupiterOne's DevOps team leverages JupiterOne to understand service coupling and deployment order, by mapping CodeRepo USES CodeRepo relationships. We also use an Insights dashboard to help answer questions about deployment order.

What is a Deploy Dependency?

A deploy dependency is a service whose outputs are used as inputs in another service (specifically Terraform outputs). Based on that, we know if service B must be deployed prior to service A. Ideally, we would like to see a relationship in JupiterOne between these two CodeRepo entities.

Where are Deploy Dependencies Defined?

We define these deploy dependencies in a dependencies.yaml file that exists in all of our deployable repositories.

Example dependencies.yaml for an account-service:

---
terraform:
- user-service
- invitation-service
- feature-toggle-service

Since the account-service USES the outputs of the services user-service, invitation-service, and the feature-toggle-service, we know that those must be deployed prior to the account-service itself.

By answering the questions above, we have what we need to automate the creation of CodeRepo USES CodeRepo relationships in JupiterOne.

Our team developed a script that parses the dependencies.yaml of our deployable repositories in GitHub, creates relationships based on the matching CodeRepo entities with the following properties:

const relationshipKey = `${mainRepo.entity._key}|uses|${dep.entity._key}`;
const relationship: RelationshipForSync = {
_key: relationshipKey,
_class: 'USES',
_type: 'repo_dependency',
_fromEntityId: mainRepo.entity._id,
_toEntityId: dep.entity._id,
deployRelationship: true,
};
payload.push(relationship);

 

Next, we do a bulk upload into JupiterOne using the Node.js Client and CLI, which is available on our GitHub.

const jobState = await J1Client.bulkUpload ({
scope: 'summary-relationships-code-repo-dependencies-workload',
relationships: payload,
});

Currently, our DevOps team runs this on a daily cadence to keep our deploy dependency relationships up to date.

The JupiterOner node.js API client wrapper and CLI utility supports uploading entities in either JSON or YAML format. For another example use case, we recommend: Using JupiterOne as a central repository for SecOps and compliance artifacts.

How to Visualize Deploy Dependencies

Now that the relationships exist in JupiterOne we can start to visualize our deploy dependencies. Here is an example JupiterOne query that shows us the deploy dependencies of the account-service example used above:

Find CodeRepo with name = 'account-service' that uses >> CodeRepo return TREE

01-visualize-deploy-dependencies

 

We also created a useful dashboard to get an overview of our deployment coupling using JupiterOne Insights.

02-insights-dashboard-deploy-dependencies

 

You can learn how to get started with prebuilt or customized Insights Dashboards here.

 

Repository Deployment Coupling

Here is the configuration we use for the first widget displayed on our DevOps team's Insights dashboard above, which creates the pie chart labeled "Repository Deployment Coupling":

Find CodeRepo with tag.AccountName="YourAccount" as f
that USES as rx CodeRepo
where rx.deployRelationship = true
return f.displayName as none, count(rx) as value

03-repository-deployment-coupling

 

Repository Deploy Dependency Direction

And here are the configurations for the second widget, the lower bar chart labeled "Repository Deploy Dependency Direction":

Find CodeRepo with tag.AccountName='YourAccount' as f
that USES << as rxIn CodeRepo
return f.displayName as x, count(rxIn) as y
order by y DESC

04-repository-deploy-dependency-direction

 

Find CodeRepo with tag.AccountName='YourAccount' as f
that USES >> as rxOut CodeRepo
Return f.displayName as x, count(rxOut) as y
Order by y ASC

05-repository-deploy-dependency-direction

 

Future Goals: Identify Direct and Indirect Circular Dependencies

These relationships have proven to be extremely useful in determining deployment order, but there is quite a bit of extra insight we could get by iterating on these relationships. For example, we could start to identify circular dependencies (direct and indirect) and with additional logic determine a specific order for services to be deployed. A shared dashboard for common understanding of deploy dependencies has been a valuable tool for our DevOps team, so we look forward to visualizing these additional DevOps and SRE use cases.

For more use cases and insight into how JupiterOne's team uses our own solution, we recommend the JupiterOne SecOps Automation Examples repo on Github.

Neal Patel
Neal Patel

Neal Patel is a DevOps Engineer who focuses on improving CI/CD, developer tooling, and observability. When, he's not on the clock you can find him getting cutoff by bartenders for pouring shots for his friends :zany_face:

Keep Reading

Mitigate CVE Risks Faster with Asset Visibility | JupiterOne
May 16, 2024
Blog
Mitigate CVE Risks Faster with Asset Visibility

Discover how JupiterOne addresses critical vulnerabilities with asset inventory, relationship mapping, and actionable insights for enhanced security.

Introducing Continuous Threat Exposurement Management | JupiterOne
April 30, 2024
Blog
Introducing Continuous Threat Exposure Management with JupiterOne and watchTowr

Introducing Continuous Threat Exposure Management (CTEM) with JupiterOne and WatchTowr: A Proactive Approach to Cybersecurity

Why Your Business Needs Cloud Asset Management
April 10, 2024
Blog
Why Your Business Needs Cloud Asset Management

Organizations are transitioning to the cloud faster than ever to keep up with the changing consumer and business climate. According to Gartner, by 2023, 40% of all

15 Mar 2022
Blog
One line headline, one line headline

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud eiut.

15 Mar 2022
Blog
One line headline, one line headline

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud eiut.

15 Mar 2022
Blog
One line headline, one line headline

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud eiut.