What’s the best thing about Salesforce? Many of us would say that it’s the ability to deliver change fast. Any Organization on Salesforce gets big benefits from the way that it supports fast delivery cycles and continuous improvement. With three new releases every year and a set of tools that makes it easy to develop your own features or deploy Salesforce’s new ones, fast, valuable delivery is what it’s all about.
But there’s one part of the delivery cycle that’s resistant to speed: testing. Or rather, manual testing. Agile might have sped up your team’s delivery cycle, but you can’t speed up your human testers to match.
“For Continuous Delivery and Agile to succeed, we need to bring manual testing from 60-80% of the testing effort down to 5-20%.”
The adoption of test automation in Salesforce is on the rise, especially at the enterprise level. But why? To understand this shift, we need to understand what problems manual testing presents and how test automation can address them.
1. Manual testing takes time. This is a big one for agile, where time is of the essence. If you’re relying on manual testing alone, this can seriously slow down your delivery cycle.
2. Manual testing is expensive. If you’re setting two testers on regression testing for a week, that’s 40 hours of human effort. Automation can cut this out completely.
“The Monotonous Tester”, Cartoon Tester
3. Manual testing is not fun. Regression testing is repetitive and formulaic by definition. Can you imagine running the same 150 tests for every new deployment? Hint: it’s boring. It’s also a waste of talent: testers enjoy exploratory, creative testing, and this is where they bring the most value. With automation you can free up your testers to do more exploratory work, which is more fun for them and more valuable to your team.
4. Manual testing is open to mistakes. Testers are humans, not robots. Only robots excel at repetitive activities. When we’re bored, we become less consistent and we’re more likely to make mistakes or cut corners. Automated testing is performance-consistent, so it’s a safer bet for repetitive tests.
So this boils down to four things: test automation can reduce time, cost, boredom and inconsistency in your testing. These are the four main reasons that enterprise Salesforce customers are moving towards test automation.
It’s simple really: all test automation does is reduce manual testing effort by taking care of those tests that need to be run regularly. Once you set up your automation, those repetitive tests will run themselves. You just need to press play when you want them – or alternatively, set them up to run nightly and just check the results in your inbox in the morning. Bingo! This leaves your non-robot testers free to conduct the type of creative, exploratory testing that brings more value.
So we’ve seen how valuable test automation can be. But why is this trend only just emerging? If automated testing was so easy to achieve, why isn’t everyone doing it?
Well, in the past, the options for Salesforce test automation weren’t so good. Traditional test automation required a developer to write tests in code, using a framework like Selenium. The developer would start by ‘teaching’ that framework all the relevant Salesforce concepts, like page layouts and validation rules. Then they would write, design and execute their tests through code. Any small change to the system under test, even a field moving on a page layout, would need a corresponding code update, otherwise tests could break. The more updates you made to Salesforce, the more updates you’d need to make in the code.
“Automation”, xkcd 1319
Coding your automation has all the drawbacks you can imagine. It’s also totally against the Salesforce philosophy. Why choose a platform that has so many declarative options and supports a rapid delivery cycle, and then invest in a large codebase that needs a lot of maintenance to support the testing of that platform?
So test automation hasn’t been widely adopted until now, and that’s down to the sheer lack of good tools. Until recently, there weren’t robust tools that could provide test automation, code-free, to the point-and-click Salesforce audience. Admin-friendly tools simply didn’t exist. Fortunately this is starting to change as the market catches up to demand.
But this isn’t a one-size-fits-all situation. You need to choose the tool that works best for your own organization.
To make this choice well, let me offer some advice.
Below you’ll find my test automation checklist. This list is designed to help you find a robust Salesforce test automation tool that can help you achieve the four main benefits of automation as I see them: reduction of time and cost, saving your testers’ boredom and increasing the consistency of your testing.
The Automation Tool Checklist
If you’re looking into test automation, seek a tool with the following features:
1. It’s code-free. As I said above, investing in a lot of code to automate your testing does not make sense for a platform like Salesforce that provides so much declarative power. Your tool should have a point-and-click interface and be admin-friendly.
2. It’s flexible. Your tool should be able to tests across different environments and browsers without requiring changes to the test case. Low-quality tools commonly hardcode things that change in different environments, like Field IDs, making your tests brittle and maintenance-heavy. Avoid this.
3. It’s smart. Your tool should be able to handle minor cosmetic changes without tests breaking (e.g. moving fields on a page layout or Visualforce page). Human testers wouldn’t get confused by this, and your automation shouldn’t either.
4. It knows Salesforce. Your tool should be able to test advanced elements like Visualforce pages and the Service Cloud console. Even if you don’t use these elements yourself, it’s a good test of product maturity. Many tools struggle with embedded tables and tabs.
5. It supports integration. Your tool should be able to connect up to other systems such as databases or your email system. This will help you to do true end-to-end testing of your processes instead of looking at Salesforce in isolation.
6. It generates reports automatically. Your tool should be able to generate reports on the successes and failures of your tests. It should also have options for running tests automatically, e.g. on a nightly basis, so that you can receive a report in your emails in the morning and just scan through. (This is normally done through a continuous integration system, e.g. Bamboo.)
7. It’s Lightning ready. Even if you’re not using Lightning yet, your tool should allow you to run tests in both interfaces. This will let you make the switch later on without dependencies.
I was a Salesforce consultant for five years. During that time, I only found one test automation tool that ticked all the boxes, and that was Provar. With these capabilities, Provar has seen rapid adoption among enterprise-level Salesforce customers as well as ISV partners, for whom regression testing is a time-consuming, costly but essential stage of delivery.
But you don’t have to take my word on this. Let me give a final piece of advice.
1. Take my checklist and make your own additions.
2. Contact us for a demo or a free trial.
3. Compare! Do your own research, find other test automation tools and see how they match up. Don’t forget to use your checklist.
I hope this guide will be useful to those who are just starting out with test automation. If you’d like to learn more, our FAQs are a great place to start.
Originally posted on Salesforce Ben’s blog on June 12, 2016.