by Aytekin Tank, founder & CEO of JotForm
What does it mean to “listen” to customers? Companies toss around this language while, ostensibly, chasing their competitors and ignoring what customers want. It doesn’t take Sherlock Holmes to find cases where technological innovation and the best interests of customers are at odds.
What are such companies “listening” for? Maybe they listen to customer service conversations with clients. Perhaps they listen to social media posts, online reviews, or forum chatter. They might even test a product with select customers and listen to their feedback.
Nonetheless, it’s easy to listen for the wrong signal or entirely miss what matters. My company JotForm almost made that mistake at what would have been a colossal cost.
I’m sharing our story and what we learned in hopes of sparing entrepreneurs from similar risks. To assess the value of innovation, sometimes we must redesign how we “listen.”
The situation.
Every year, I come up with a BAUG for JotForm: a Broad, Ambiguous, Unmeasurable Goal. We use it to guide our long-term innovation plans so that we don’t get stuck in reactive, short-term development projects. Our 2017 BAUG was, “Reinvent forms. Reinvent JotForm.” Almost all of our daily web traffic went to forms, so this goal made sense.
We ran three “Hack Weeks,” five-day sprints in multiple cross-functional teams, to tackle different aspects of redesigning forms. After a fourth and final hack week, we were ready to test our new “JotForm Cards” in August 2017.
Unlike traditional forms that place multiple fields on one page, Cards put one question on each page. They’re visually appealing and can feel quicker because you deal with one question at a time and not the other 15 awaiting your attention.
Given all the work we put into JotForm Cards, we thought we’d knocked it out of the park. Back then, about 3,000 people signed up for JotForm daily, so we ran a simple A/B test. 50 percent would get traditional forms, and the other half would get Cards. We then measured the percentage of users who became “active,” meaning they used the form to receive submissions.
The Card group was 30 percent less likely to become active! What was going on?
What are we really testing?
We could have concluded that users hated Cards because the data supposedly said so. However, we’d put so much effort into creating Cards and couldn’t just abandon them. One test isn’t conclusive. Until you’ve tested multiple variables, you don’t know what you’re “listening” to.
We tried new background colors and retested. We failed to find correlations between industries and card preference. We added a progress bar to Cards, hoping that would improve the Card experience.
Ultimately, we tested eight variations without affecting the test result. We even ran email surveys, which found that most people didn’t like one question per page.
Were JotForm Cards a flop? Were we ignoring what customers plainly told us?
A new test design.
In September, we held a brainstorm session to rethink our test. One designer had a flash of insight, which he depicted with a Venn Diagram. What if people had a strong preference for one type of form over the other, but our experimental design prevented them from having a choice?
We created three groups for our next test. 1,000 people got traditional forms, 1,000 got JotForm Cards, and 1,000 could choose between the two. A week later, the ‘both’ option hadn’t performed any better than the other two.
Then, another designer raised a question: What if people didn’t understand the options? Listening goes both ways, after all.
In the eight test, people who received both options saw two icons they could click. One said, “Create Classic Form” and the other said, “Create JotForm Card.” Both had simple illustrations. We revised them to say, “All Questions in One Page” and “Single Question on One Page.” And we waited.
The power of choice.
The ninth test had resoundingly different results. New signups who could choose between forms activated at a 40 percent higher rate. We saw a 50 percent increase in paid users among the cohort that had a choice.
More people liked traditional forms than JotForm Cards, but offering a choice was the only way to capture the preferences of both groups. It seems so obvious in retrospect, yet we had become trapped in believing that Cards were superior and intended to replace traditional forms with them.
Our real innovation was not to make Cards – it was to discover that giving users options is more powerful than any individual option. “Listening,” in the truest sense of the word, was recognizing that we couldn’t presume to know what users would want given two choices. We couldn’t superimpose our preferences. We had to step back and say, “You know best for you.”
Is choice a form of innovation?
Listening to customers took nine attempts in total. It was as if we were a radio station operating at the wrong frequency, so we blamed our music selection instead of the underlying problem. Then, eventually, we found the underlying problem, but there was no guarantee we would.
Of course, nothing was wrong with Cards. The situation led to a question for me: Is choice itself a form of innovation? If we produce three more types of forms, would we see even happier users? Or would choices backfire at some point?
Without listening to consumers, we tech entrepreneurs assume that they want advancement and change. Everything should be more beautiful, easier, faster, and so forth.
Yet, maybe control over change is more important than change itself. Not every user wants to be dragged away from a familiar interface just because the tech company assumed they wanted something “better.” Remember how many people hated the Windows 10 interface and installed third-party software to preserve the old Start menu?
Innovating at high speed isn’t necessarily great for customers. Innovating to beat competitors could be even worse because what they release doesn’t necessarily change what your users want. Question what you’re really listening to before imposing change without choice.
Aytekin Tank is the founder & CEO of JotForm, the first and only full-featured online form building tool that is completely mobile friendly. JotForm allows anyone to create forms and collect their data, without writing a single line of code.
