Team Gearset has returned from a fun and frenetic week at Dreamforce. As always, we had a great but exhausting time - we spoke to some of our fantastic users, we spoke to lots of people who are facing problems that Gearset might be able to help solve, we partied a lot less than we’d planned to due to aforementioned exhaustion, we tweeted some atrocious U2-related puns and we found out about all the cool new things that we can expect to see from the platform in the coming months, all to the backdrop of the world’s craziest party tech conference. One of the big talking points was Salesforce DX - Salesforce’s new developer experience.
Salesforce DX is a new paradigm and accompanying set of tools aimed at providing a more modern user experience for Salesforce developers. Of course, given that we’re in the developer tools business and operating in a similar space to Salesforce DX, we met with a good number of folks who were interested to hear what we thought about it.
In this post, we’ll discuss what we’ve learned about Salesforce DX, what we think it has to offer, and where it currently falls short. Note that Salesforce DX isn’t available to try yet, so everything that follows is based on what we’ve learned from sessions and demos at Dreamforce, and from talking to other developers. As the functionality is made available, we’ll share more details based on our hands-on experiences.
Why Salesforce DX?
One of the biggest causes of tension faced by development teams is keeping up with inevitably shifting business requirements. Salesforce counteracts this by enabling a much broader group of people than other platforms do to safely and confidently make changes to applications, empowering organizations to be more agile and to focus on business outcomes. Indeed, the slimmer process of no- or low-code development is part of what affords this greater agility to teams building on top of Salesforce - it’s quicker and easier to get things done.
There are, however, some problems that go beyond the capabilities of a no- or low-code solution. As business needs grow and become more complex, solutions tend to migrate away from the no-code approach, through low-code towards a more full-fat, traditional development process. Some business problems simply require bespoke logic, which Salesforce understands and amply provides for in the form of Force.com, but a programming environment to enable this bespoke logic isn’t sufficient - to take advantage of these features, more traditional developers and development teams are required.
These more traditional development teams have their own expectations - the benefits of moving quickly and pushing out releases with a frequent and regular cadence are well documented, but along with it comes additional process. Thankfully, across most languages, stacks and platforms, the tools to support this approach have developed accordingly. These tools not only improve the day-to-day lives of developers, they also demonstrably improve the ability of development teams to deliver business value more quickly. As a result, these features are expected by developers and their management hierarchies alike.
Salesforce has done a fantastic job of empowering #AwesomeAdmins and Trailblazers to be impactful in a way that no other platform can, but at the same time fallen behind when it comes to catering to the more traditional development team. This is where Salesforce DX comes in.
What is Salesforce DX?
At its core, Salesforce DX is a suggestion that the platform’s ongoing growth and success is, in part, dependent on its ability to sustain more traditional development teams. It also represents a commitment from Salesforce to put effort into providing the requisite tools and support. So what does this look like in practice?
A new philosophy
Traditionally, the org itself has often been its own sole representation. Dev-heavy teams who’ve come from other platforms, or those with more stringent requirements and rigorous practices, use tools like Ant, Eclipse and MavensMate along with the command line to version control their metadata, thus introducing a second representation of the org’s configuration. Although advisable, this practice has been made challenging by the limitations of the metadata API, and first-party tools in this area have been in short supply.
Throughout the announcements of Salesforce DX, one phrase has been pervasive - “version control is the source of truth”. The message is loud and clear - to work and collaborate effectively, developers need to use source control, and for this to be practical it needs to act as the ground-truth representation of an org.
For developers, this is a great decision. Version control brings with it a host of benefits in terms of collaboration, auditing and disaster recovery, but perhaps equally as important, it’s a vital component of a truly agile release process. If the ultimate goal is to be able to build features quickly and put them in the hands of the users that need them as soon as they’re available and with minimal risk, then version control is the first stepping stone.
The only question hanging over this philosophical shift is what impact this will have on those developers and admins who are less comfortable using the more developer-centric command-line tools. For a shift towards version control as ground truth to be successful, it needs to apply across the board, in order to avoid changes to orgs that aren’t tracked by version control, for example.
New and improved tools
CLI
The new SFDX CLI bundles existing features found in the current Force.com CLI Heroku add-on and the Force.com migration tool with some new features. There’s not a huge amount that’s new here - retrieving and pushing metadata from and to and org has been available in the Force.com migration toolkit for a long time, for instance - but the CLI pulls all of this functionality into one place, and the parts that are new are exciting. It’s also open source and extensible, so going forward you can expect to see its capabilities extended by both Salesforce and the surrounding community.
The big new features to look out for are:
force:data:import- This imports data from a json document. Particularly useful when employed in conjunction with:force:org:create- Theforce:orgcommands are used to manage the new Scratch Orgs.
Scratch orgs
Scratch orgs are possibly the most impactful new feature in enabling the version-control-as-source-of-truth workflow. Scratch orgs are ephemeral orgs that can be trivially created and destroyed with the force:org:create and force:org:delete commands. Rather than creating a DE org or using a sandbox, and attempting to keep that synced to a corresponding source control branch, a developer can now use existing tools like Eclipse and MavensMate, and when the time comes to “run” the changes spin up a new org and push metadata to it.
sfdx force:org:create -f config/org-definition.json -w
sfdx force:src:push -w
sfdx force:data:import --plan data/org-sample-data.json
The above will create a scratch org based on the definition of the org in org-definition.json, push the contents of your cloned version control repo to the new scratch org, and import sample data defined in org-sample-data.json. This means you can go from version control to a running representation of your org in just a few seconds.
This has a profound impact on the standard development workflow. Because these scratch orgs are ephemeral and quick to spin up, developers will no longer need to maintain and sync orgs that map to their version control representation for day-to-day development work. Switching branches will now be trivial, and testing your changes will be as simple as clicking “run” in your favourite IDE, which can spin up a scratch org behind the scenes.
Of course, scratch orgs don’t make sandboxes obsolete. For more long-lived environments like dev integration, staging, QA, and so on, a sandbox will still be the de facto representation. This also means that whilst scratch orgs will be fantastic for the minute-to-minute development experience, they won’t be overly helpful in taking that representation from version control and turning it into a viable patch that can be applied to an existing environment. Pushing metadata to a completely empty org is one thing, but building a viable deployment package and pushing that to an existing org will still be fraught with the same difficulties teams face today.
Force.com IDE
The Force.com IDE is being revamped with an updated Eclipse plugin built on top of the new SFDX CLI. The main new features you’ll see here are:
- Most of the key functionality from the command line is directly exposed as a command palette in the IDE (scratch org management, pushing metadata, pulling metadata, running tests, etc.)
- A shortcut to jump straight to the lightning app builder in an embedded browser, with the correct context from the corresponding component metadata
Being built on top of the CLI means not only that you know exactly what’s being run behind the scenes when you click around the IDE, but also that changes and enhancements to the CLI from both the first-party and the community will likely be more easily incorporated into the IDE going forwards.
Heroku pipelines + CI
All of the tools we’ve mentioned above can be leveraged by vendors to build CI solutions on top of Salesforce, like Bitbucket pipelines, using a CI server like Jenkins with the new command line tools, and of course Gearset CI.
There’s also a new player in the Salesforce CI game - there are several recent and new additions to Heroku that facilitate building out CI for your orgs. In short, via the Heroku tools it’s now possible to define a build pipeline from dev through to production, creating scratch orgs to make it easy to review changes. Again, some of the constituents aren’t currently publicly available just yet. It’s also unclear how pipelines will handle the common challenges of migrating changes between environments that we touch on in the next section, but we’ll cover this in more detail as we find out more.
What Salesforce DX isn’t
It’s not the finished product, yet. In fact, it’s not available to the vast majority of users and there’s currently no hint of a release date for GA. It’s also not a silver bullet to solve all of your release management problems, from what we’ve seen so far.
Current challenges resulting from limitations of the structure of metadata and the metadata API in terms of supported types, idiosyncrasies, and dependencies will still persist. Want to deploy changes to a flow? The existing gotchas will likely still apply. Deploying changes to profiles? You’ll still need to dive into the xml. Metadata incompatibilities because of different org versions? You’ll still need to handle these. The new tools will provide a vastly superior alternative to the popular combination of Ant and Jenkins, but the bigger issue - that building a valid deployment package is difficult, error prone and time consuming - will still persist, at least for the time being. The few issues called out above are the tip of the iceberg.
The other major shortcoming is that it’s presently a one-sided take on the developer experience. Building on top of Salesforce has never been solely about the hardcore developer. Arguably this group makes up a fairly small portion of the broader cohort of people building things with Salesforce. Salesforce DX, as it stands, helps the more traditional developer work from source control in a way that’s familiar to them, but it’s still unclear how they’ll collaborate with their less technical counterparts.
Gearset and Salesforce DX
So where does Gearset fit in alongside Salesforce DX? Firstly, on the surface there are some similarities. For example, when released Salesforce DX will provide new tools to make it easier than it’s previously been with free tools to set up CI builds for your Salesforce orgs, whilst Gearset offers CI out of the box. Similarly, Salesforce DX and Gearset both make it easier to get your metadata under version control - Salesforce DX with its new command line tool that supersedes Ant, in combination with the git command line, and Gearset by allowing you to run comparisons and deployments directly against your version control system or local disk.
Integrations
More important, however, are the ways in which Gearset and Salesforce DX complement each other and can be used together. In the example above, Salesforce DX will help improve the workflows of the more technical by making it easier to pull their metadata to their local disk and to test and debug their changes via scratch orgs. Gearset, on the other hand, will enable the technical and less technical alike to adopt a similar workflow. By making it straightforward to sync changes back and forth between an org and your version control system in just a few clicks, with Gearset somebody entirely unfamiliar with git can run a comparison and deployment between their sandbox and an arbitrary git branch, just as they would two orgs, empowering the less technical to work with the same “version control as source of truth” philosophy as the developers in the team.
Depending on the maturity of your release management process, understanding the shape of your org and the relationships between the objects within it may come as a precursor to pushing changes and setting up CI. Gearset’s full org diff complete with dependency analysis gives you the insight you need into the differences between your environments. Once you start building deployment packages Gearset’s problem analysis will help you build deployment packages that are successful on first push, avoiding the familiar cycle of build-deploy-fail-repeat.
When the time comes to set up CI, you might choose Heroku CI, Jenkins, TeamCity or Gearset, for example. Using Gearset as part of this process means the aforementioned missing dependency and problem analysis will kick in and offer to fix a whole host of issues - whether you’re pushing to an org or directly to version control - so that what’s in your version control system is much more likely to be deployed successfully by your CI setup. After all, attempting to continuously deploy fragile packages will result in lots of frustrations and failures - automating a broken process is still a broken process.
You might choose Bitbucket pipelines, Heroku CI or Gearset to promote changes between environments (although Gearset does offer the very helpful org delegation and user permissions for controlling which users can perform what operations!), migrating your deployment package through to production. Gearset can then monitor your production org and alert you to any unexpected changes - because sometimes hotfixes need to happen - and silent test failures, making sure you’re always aware of the state of your orgs and tracking a full history of changes.
Going forwards
The above represents just a few examples of how Gearset’s existing functionality can be used in unison with Salesforce DX. As we get our hands on it ourselves, you can expect to see several more ways in which we’ll take integrations to the next level.
Taking one simple example of a possible future extension - that of running tests. Taking advantage of the new scratch orgs, Gearset will be able to monitor all of your git branches on a particular repo and spin up a corresponding org to run tests whenever a change is detected. The resulting status could be attached to the branch as an indicator of whether it’s ready to be merged. This is just one example - there are several other really exciting ways that Gearset and other third-party vendors could take advantage of Salesforce DX.
Salesforce DX is the first step towards a modern process for Salesforce developers with a slew of new and useful features. More importantly, it’s fundamentally underpinned by the right philosophy.
There’s no suggestion that Salesforce DX will help devs, admins and release managers tackle the most difficult problem - building valid deployment packages - which is largely dictated by the nature of the underlying metadata and the restrictions imposed by the metadata API. Gearset shows you exactly how your orgs differ and makes it easy to selectively compose changes into a deployment package. We continually commit significant effort towards building out our intelligent dependency and problem analysis engine, meaning that over time we’ll catch more and more issues with your deployment package before pushing it to Salesforce. We also understand the realities of projects in the wild - changes inevitably happen outside of the release management process, and Gearset provides monitoring, alerting and history to keep you appraised of these changes before they have a chance to disrupt your process or, worse, your business.
Explore more
For teams looking to take the next steps in DevOps and create a truly end-to-end release process encompassing developers and admins alike, Gearset can get you there — try a free 30-day trial today. If you want to learn more about Salesforce DX and how it can fit into your DevOps process, download our free whitepaper.
