All products featured on WIRED are independently selected by our editors. However, we may receive compensation from retailers and/or from purchases of products through these links.
Jason Huggins sells a robot that mimics the touch of the human finger. But it’s not what you think. Known as Tapster, his contraption is a means of testing new software applications on the Apple iPhone and other touchscreen devices.
Software developers often use automated programs in testing their new applications, but Tapster takes the idea much further. It’s not a software program. It’s hardware — and it has a finger (see video above).
Well, it’s really an iPad stylus. But it acts a lot like a finger, and you can program the thing to behave the way you want it to. Huggins will sell you one for just under $1000. Or you can build your own. He has also open sourced the Tapster design using BitBeam, a collection of parts that can be assembled with 3D printers.
In addition to using to software programs for automated testing, app makers typically employ people — real people — to manually test their new creations. But Huggins believes that tools like Tapster will push these people back onto the streets. “About 70 percent of the money spent on testing is in manual testing. About 30 percent is automated,” he says. “[But] I think that trend is changing. I think the trend is towards more automated testing.”
Typically, app makers rely on real people to test things that can’t easily be automated or emulated, but it’s not exactly glamorous work. “Manual testing has been viewed as the lowest form of life on the dev cycle,” says Bryce Day, CEO of Catch, a company that makes applications for managing software testing. “If you can’t code and you can’t write specs, you can hopefully catch bugs as they roll out.”
But it’s still important — partly because automated testing tools often lag behind the invention of new development platforms. In the early 2000s, for instance, more companies began building web applications, and although there were certain automated tests that could be run on the net servers, manual testers were often needed to test interfaces that showed up in user web browsers.
That’s why Huggins, the CTO of Sauce Labs, created Selenium. Now almost a decade old, Selenium is an open source framework for running automated tests on web applications. It lets developers test many parts of an application in any browser, including Internet Explorer, Firefox, and Google Chrome.
But then the iPhone came along, and this presented yet another challenge. Selenium was designed to simulate mouse clicks and key presses. There are many more ways to interact with modern smartphones, such as pinching or swiping the screen or shaking the whole phone. And at least initially, Apple didn’t offer API, or application programing interface, for building testing tools. That meant you needed manual testing for mobile apps.
Although some automation tools are now available, such as Huggins’ Appium, things like gestures or movements are hard to simulate. Hence Tapster, which Huggins thinks will bridge the gap between manual and automated testing.
Tapster started as joke: a simple robot that could play video games like Angry Birds. He created the original version of the tool, BitBeamBot, about two years ago and started showing it off at conferences. Then, about a year ago, he put it up for sale on the homebrew hardware marketplace Tindie. Soon, the calls came: he hadn’t put a price on the machine, and many people wanted to know how to buy one.
After a while, he realized it could also be used for testing. After all, smartphone manufacturers and carriers often test their wares with robots.
That led to the creation of Tapster, which is more expensive, and designed specifically for phones, not tablets. It adds several new tools that BitBeamBot didn’t have, most notably a base for holding phones still. Huggins says the original BitBeamBox was just a hack, but he’s put a lot of work into relearning trigonometry to calibrate Tapster. For test validation, the robot can be integrated with both Selenium and Appium. Alternately, a camera can be mounted and the open source machine vision system OpenCV can be used to compare screenshots to determine whether a test passed or failed.
Huggins says that although test preparation company Kaplan is experimenting with Tapster, no one is using it as part of their testing workflow as of yet. “As with any new technology, people who have Tapsters are kicking the tires and learning how to program it,” says Huggins. “It’s still early days in the robot-powered testing revolution.”
But he thinks that revolution is coming.
Day, on the other hand, argues that manual testing shouldn’t go away. “Internally, we did some research and determined that going with an all-automated system would actually cost more,” he says. “If you want to go automated, you will need multiple tools, and those tools require different talent. The cost is actually quite substantial.”
Day says about 80 percent of the cost of automated testing lies in creating the test plans. Catch’s flagship product, Enterprise Tester, actually automates this part. It takes specs created by analysts and software architects and automatically generates test plans. These can then be handed over to manual testers to run. These test plans can be used over and over, even as the code base and interface changes because human testers can adjust accordingly in a way a test script cannot.
Huggins agrees — up to a point. He thinks manual testing will continue to be useful, but he says that automated testing isn’t just about cost, it’s also about speed. “Agile development makes developers want to go faster, but testers can’t keep up,” he says. “People who want to slow down the process, whether that’s a database administrator or a tester, those people are being phased out.”
So what can black box testers do to stay relevant in the age of robot testers?
“If you’re in manual testing, you should learn to program,” Huggins says. For those who can’t or don’t want to learn to program, he suggests getting involved in analytics and A/B testing, which is usually the province of marketing departments. “They have lots of data that could be used for testing,” he says. “There’s an opportunity for someone to take the role of middleman between software development and marketing analytics.”
