How to keep your Salesforce environments in sync
David Runciman on June 18th 2021
Struggling with Salesforce orgs that are out of sync? You're not alone. Respondents to our State of Salesforce DevOps 2021 survey reported that keeping their orgs in sync was a significant challenge. The good news is that several other respondents said their DevOps workflows had helped solve this problem. In this post we'll take a good look at how to keep your orgs in sync: why it matters, the challenges, and the best solutions.
Why Salesforce environments get out of sync
Your Salesforce environments are constantly changing as you build features, deploy them along your release pipeline, and finally release them to your production org. Over time, differences in environments begin to arise, both in terms of data and metadata.
The whole point of sandbox and developer orgs is that you can click around, building and testing customizations without worrying too much about the impact. But unless you deploy everything you build, the changes you try out but then decide not to use can quickly clutter up your sandbox. At the other end of your release pipeline, any hotfixes made directly in production won't be reflected in upstream environments.
Data is constantly being added to production by your end users, and this includes new kinds of data in different formats for various custom fields. As a result, the datasets in your testing environments quickly get out of date.
Why does syncing up orgs matter?
Keeping your environments in sync is really important. It makes your entire release process much easier, preventing issues with releases no matter what that process looks like, and it's particularly essential for DevOps automation.
More reliable testing
If you're testing work in a sandbox that isn't in sync with production, then you won't discover every problem before releasing those work items. There might be an edge case that causes your work to fail in production due to metadata or data that wasn't in your testing environment. As one survey respondent says, keeping orgs in sync means you can 'catch issues earlier'.
If you're using Gearset's powerful comparison tool to see the metadata differences between your orgs, you don't want to have orgs that are massively out of sync. Otherwise the comparison will show you a huge number of differences, and you'll need to search carefully for the changes you really care about.
Unreliable testing and an overload of information about differences between environments make it more difficult for you to deploy between out-of-sync environments. And the likelihood of deployment success is lower because the metadata in your target environment may not be the right shape to receive the deployment package.
Adding automation to your release pipeline using continuous integration jobs relies on environments that are in sync. CI jobs are designed to automate frequent deployments of incremental changes. The more metadata a CI job is trying to deploy, the more likely it is to fail.
Survey respondents on syncing challenges
"Keeping multiple orgs in sync is a real challenge."
"A large project with consultants has made it difficult to keep environments properly in sync."
"We have problems with sandbox and production org desynchronization."
"Our UAT org is way different from PROD!"
"With multiple teams releasing separately our sandbox refresh schedules conflict."
"We're struggling to refresh and get data into our partial sandboxes on a set time frame."
Options for solving the problem
So how do you get environments in sync? Essentially, you need to make sure your upstream environments have the canonical version of your metadata, and that testing environments have recent, realistic data. Salesforce's sandbox refresh is a useful tool, but it has limitations. Teams using Gearset as a comparison and deployment tool can improve the process. And teams using Gearset for DevOps, with version control and automation in their workflow, can take things even further. Let's build up to that step by step and look at each of the tools you'll be using.
Sandbox refresh in Salesforce
Refreshing a sandbox in Salesforce is the traditional way to get it in sync with your production org. Running this process brings the sandbox's metadata in line with production and, depending on your particular sandbox setup, it can update the data as well. This can be an effective option, especially if your orgs are really out of sync.
Refreshing a sandbox does come with some dangers and drawbacks. First and foremost, there's no visibility into what you're overwriting in the sandbox, so you might accidentally lose development work that hasn't yet made it to production. The idea, of course, is that sandboxes are refreshed at the start of a release cycle, but that doesn't work for high-performing teams releasing up to multiple times a day.
Salesforce can take several days to complete the refresh, during which time you won't have access to your sandbox. Salesforce also limits how often you can refresh your sandbox. And the refresh isn't strictly a migration to your existing sandbox; it tears down the old sandbox and replaces it, which means your sandbox will sometimes 'move' to a different Salesforce instance, and you'll need to update your Gearset login if you use your sandbox as authentication. Finally, there are quirks in Salesforce that mean your refreshed sandbox isn't always actually an exact copy of production. More on this below.
If you're a small team with a slow release cadence and a simple release pipeline of a few orgs, then Salesforce's sandbox refresh will meet your needs - but, even so, you'll be doing very well if you manage never to lose any work. Most teams need the help of more powerful tools.
Gearset's comparison and deployment engine
Gearset can immediately improve your sandbox refresh process, even if you haven't yet adopted DevOps processes. It gives you more visibility and control, and can sometimes help you sync your environments without needing to use Salesforce's sandbox refresh. Here's how:
1. Compare your orgs
Run a comparison in Gearset with production as the source and the sandbox as the target. Make sure you select Compare all in the metadata filter, and bear in mind that this comparison will be slower than usual because Gearset's is comparing all of the metadata in your orgs. Once you've reached the comparison view, you'll quickly get a sense of how badly your orgs are out of sync.
2. Analyze the differences
This kind of comparison with production as the source is the reverse of a typical comparison where you're looking to promote changes along your pipeline. In this case, the New items tab shows you any metadata in production that isn't in the sandbox, i.e. any changes that have been made directly in production as a hotfix or bypassing the official deployment process. The Deleted items tab shows you metadata in the sandbox that isn't in production.
The Changed items tab is trickier; it shows you the metadata items that are in both environments but have been modified or differ in some way. If your team has been following the right development process, production should have the 'correct' version of this metadata, but be sure to check that there isn't any development work in the sandbox that hasn't been deployed yet. Gearset's filters in the Changed on column will help you identify the most recent work.
3. Decide whether to refresh
If your orgs are badly out of sync, Salesforce's sandbox refresh is probably your quickest option. Remember to rescue any changes in your sandbox first, if you spotted anything in the comparison view that you want to keep. After you've refreshed the sandbox, run another comparison in Gearset. A refreshed sandbox isn't always totally in sync with production, and Gearset can help you get them closer to being in sync.
4. Deploy metadata
If your orgs aren't too badly out of sync, you could select all the changes in the comparison view (assuming you've not identified anything in the sandbox you should keep) and deploy them using Gearset. The larger a deployment, the more likely it is to run into problems, so you might want to break things down into multiple deployment packages.
5. Deploy data
Use Gearset's data deployment tool to deploy data from production to the sandbox org. In order to protect sensitive data and PII, use Gearset's data masking, which replaces the records you deploy with realistic data for testing without compromising on security and compliance.
Survey respondents on successful syncing
"More easily syncing environments is a huge plus of DevOps!"
"DevOps has reduced the number of items lost in a refreshed sandbox."
"We use change monitoring to identify production-only changes that we sync down to a sandbox if needed."
"With Gearset we can keep the lower environments more in sync."
"As well as saving time in deployments, Gearset also helps us keep our orgs in sync."
"We save hours with the ability to refresh sandboxes more often, since we don't worry about losing important work."
Gearset's DevOps toolkit
As the above quotes from the State of Salesforce DevOps survey show, teams that adopt DevOps practices find it easier to keep their orgs in sync. So what is it about DevOps that helps these teams to manage this challenge?
Version control, which underpins the DevOps process, is designed to keep things in sync. Teams using version control have a new source of truth: not production, but their main branch in Git. Developers can create a new feature branch from the main branch, commit work to this feature branch, and then ultimately merge it back into main after testing and review. Everything in Git is synced up with each cycle of that workflow.
Using version control in your Salesforce development process doesn't mean your orgs will never get out of sync. But since DevOps is about small, frequent releases, it's less common for chunks of development work to be forgotten and left in a sandbox.
For teams using a DevOps workflow like the one shown below, it's important to make sure that upstream environments are kept in sync with main, which acts as the source of truth. When you need to refresh a sandbox, use Gearset to run a comparison between main and production first, so you can identify anything unexpected in production. Setting up a change monitoring job for production will also help you to keep an eye on hotfixes or changes that might need to be deployed back to main.
Some high-performing DevOps teams have introduced automation to their syncing process, setting up continuous integration jobs that deploy specific metadata types back through their pipeline. This can be an effective way to cut down the time spent on syncing, although you shouldn't automate a process if you're not completely confident it will always work in a beneficial way.
Gearset is already helping teams to sync up their environments with less difficulty and refresh their sandboxes with the confidence that they're not losing work in the process. But there's more to come. In the next few months, Gearset will deliver some exciting product updates that make keeping environments in sync easier. Watch this space!
Get rid of that syncing feeling
If you're new to Gearset and you'd like to try syncing your orgs, you can start a free trial today and get full access to all of the tools described in this post. If you're already a Gearset user but your current license only gives you access to metadata tools and you'd like to find out about Gearset's data add-on, get in touch with us via the live chat on this page.
To find out more about how teams are overcoming a whole range of challenges with improved DevOps processes, download a copy of the State of Salesforce DevOps 2021: