Podcast Transcript: The ROI Of Split Testing

Enrollment Management Round Table with Enrollment Resources Transcript

This is a transcript of the edited recording of our podcast episode on The ROI of Split Testing at your School.

Listen to the podcast here.

Announcement:

This podcast is brought to you by Enrollment Resources, innovations in enrollment management. Learn more at www.EnrollmentResources.com.

Gregg Meiklejohn:

Hi everybody, this is Gregg Meiklejohn, with Tom King, a C-level member of Enrollment Resources. Good morning, Tom. How are you?

Tom King:

Fantastic, Gregg. I’m looking out, and it’s a nice, sunny day here, which I think is better than where you are.

Gregg:

Yes. There’s a big snowstorm here, that we rarely get. It’s fun to watch people who never drive in snow try to do it, it’s looking the panic, and the fear in their eyes, as they’re driving down hills and stuff. So, folks who are in cold parts of North America, feel free to mock us out in Victoria, British Columbia.

Gregg:

So, if I think people who are driving on these icy roads had an opportunity to, through trial and error, learn how to go down a hill, I think they would be less panicky, and they would do a much better job in their driving. So that the theme for today, hey Tom, is talk about the secret sauce of split testing. What do you think?

Tom:

Sounds fantastic, let’s go.

Gregg:

Okay. We’ve done these talks before, where we’ve gone into an audience of the business of education people and said, “What is the most important employee that sits within a marketing and admissions department?” So, we went through, we talked to the Director of Admissions, we talked about the social media people, we talked about media buyers, lead generators. We talked about graphic designers, who create look and feel, we talked about copywriters.

Gregg:

Let’s go through each one, and talk about why they could potentially be the most important person, within Mar-Comms, and lead gen, and sales. So, a graphic designer, there’s a legitimate argument that create graphic design, look and feel, might be the most important person. You want to add to that?

Tom:

Oh, sure. They’re definitely important, on the marketing side, to craft a look and feel that your company wants. I don’t want to belittle here, but that is something you can outsource. If you didn’t have it, you could still do a halfway decent job of creating marketing, through outsources, or a lot of free services that may be available out there.

Gregg:

Yeah. There’s themes for websites you can pick up for $100. But, the look and the feel of how we visually stand, even with emails and whatnot, layout and have, it’s very important.

Gregg:

Let’s talk about media buying. Within education, there are these ad agencies, and they go off and work with directories, and they bring leads to schools. Then, the Admissions Reps work these leads. So, admissions people, you’re an expert in this area, they need a volume of leads, or inquiries we’ll call them, to have a shot at converting some of those into students, which then drives an intake for a school.

Gregg:

Those in selling advertising, they have a value, right?

Tom:

Certainly. If you’re an advertising or marketing company, and having someone in there that can buy for your clients, it’s absolutely a must. But, if you’re the client, you don’t really need to have your own buyer. But, for these marketing companies, it’s an important role.

Tom:

However, many of the companies out there can support you, with their own teams, even if you don’t have somebody.

Gregg:

Exactly. Then, copywriting is all be persuasion, right? But, challenges with copywriters is if they have a mediocre offering that they have to persuade people about, then you’re only as good as, really, the offering that you’re writing about, unless you lie. Copywriting, for many people, will say copywriting is at the center, it’s the center skillset for marketing and sales.

Gregg:

Then, social media is really hot, it ties in with SEO, everybody talks about social media. It just reminds me of when you take a firecracker, and you unroll it, and you spread it out on a table, and then light the loosely packed saltpeter, it just sparkles and crackles, and makes noise. But, really doesn’t do anything, it doesn’t blow anything up, it doesn’t really make a noise. I’m of the view that social media, while it’s the branding tool for schools, if not done correctly, you’ll just be kind of meh. People do it, because they’re afraid not to. I don’t know, what do you think about that?

Tom:

Yeah, certainly if it’s not done right, it can certainly impact you. And, it’s a difficult thing to keep up with. There’s so many things changing, constantly, in the social media landscape, just trying to keep up with what’s new, and what’s out there.

Tom:

And as you mentioned, it can be a really good branding tool, but a lot of people just don’t use it correctly. They use it as a lead gen tool, or something like that. They just don’t really understand the full uses. Unless you really have a good plan, a lot of times you’re just throwing stuff against a wall, and seeing what sticks out there.

Gregg:

Yeah. People use the word Jello, throw Jello on the wall.

Gregg:

I guess, really it’s setting up for this, is that the most important skillset that sits inside marketing and admissions is the Director of Split Testing, I’ll call it. It is a rarely used job description, and it should be the most important job description in the entire department. You might have some thoughts on this, hey?

Tom:

That’s really been the hallmark, I think, of what we do, is when you start to talk about conversion rate optimization, and lean management, and process improvement, the core of that is finding the leaks, and testing out different ways to improve those, or different ways to optimize what you’re doing.

Tom:

For most companies out there, it’s a critical piece, and it’s often overlooked because of all these other shiny objects, as you mentioned, that people want to jump into. “Oh, the social media, TikTok is hot now. I’ve got to get on TikTok, and I’ve got to get on Instagram. We’ve got to do this.” When we just ignore the web page that we have out there, that somebody wrote, or we wrote, but we haven’t improved it in three years, and it’s the same website, or same web page.

Tom:

Someone that can actively take a role in process improvement testing, of all the different areas, and not just web page. All those things, as we know, add up.

Gregg:

Yeah. I guess what’s interesting is that the HR, the labor market is such that you can’t really … It’s getting tighter, and tighter, and for smaller schools, or medium sized schools, attracting top talent is really, really difficult. You either find somebody whose either mediocre in their skills, and they’re getting poor direction, or you find young people that really need skill, but don’t really know what to do. Then, you’re stuck with meh, mediocre performance within all the aspects of what you need to do in the school.

Gregg:

But, if you have somebody whose dedicated to pushing people to split test, trial and error at it’s core, you have tremendous opportunity. So, like with the graphic design person, they layout a scheme, a visual scheme, and what you can do is teach them how to just use renditions. In marketing, we call it beating the control, use renditions of the different visual packages. Then, what will happen is through trial and error, something will pop and really take off. What’s interesting is that the graphic designer, nor the Trial and Error Manager, really don’t have a clue as to why, but it takes off.

Gregg:

So, what happens is through trial and error, you can stumble upon, if you will, really interesting improvements in performance. Like we’ve learned, for instance, with what we do that if you put a caption under a photo, it will actually get more people to fill out a lead form. We’ve learned, for instance, that if you use … we’ll call them relatable people, average looking people, instead of the gorgeous ad agency supermodels used out of photo banks, actually more people will fill out a lead form. It’s an interesting notion, hey Tom?

Tom:

Absolutely. It’s funny you bring that up, because we’ve got a client mastermind webinar this afternoon, and a portion of what we’ll be going through is some split testing results, from our end, on things.

Tom:

One of the things that really stood out when I was preparing the deck and everything for that, on split testing, today is simply what the dollar value gains could be. I think from the two tests that we put together, an average school could be generating somewhere in the neighborhood of, I don’t know, $190,000 to $200,000 a year, from these two minor tweaks to, in this case, a landing page. That’s a significant amount of money.

Tom:

I don’t know that a graphic designer is going to have that impact on ROI. Again, I’m not belittling the position, my daughter does graphic designs in marketing, and I think a fantastic design can certain improve your brand, but someone who can split test, and come up with different concepts can have a much greater ROI impact on the company, than many of these other roles, and many of the other things that we’re doing. So, sometimes it’s not just the buy, the media buy, or the design, or the actual copy, it could simply come down to, hey, the button’s in the wrong place, or this color is not quite right.

Tom:

A designer might think, “Our logo is purple and blue, so we need to have a purple and blue button.” But, these split testing team may say, “You know what? An orange button works twice as good, and will get us twice as many leads, which is going to double our sales in a year.” Having that impact on the bottom line is fantastic. I don’t know any other position within the company that can have that kind of an impact.

Gregg:

So, your daughter’s what, 22 I believe? She has some graphic design skill, but she’s a kid, right? If she was fortunate enough to have a manager above her, who dedicated to pushing her gently on trial and error, whether it be design, or really, what’s more important is layout, colors, widths of paragraphs, you name it, and she could methodically, through trial and error, start testing. Then in a short order, your 22 year old daughter could become a top notch graphic designer, just simply by going through this myriad of testing.

Gregg:

The same can obvious be said for copywriting, hey? Trying different turns of phrase, different words. There’s a real art around split testing with copywriting. Most writers are out of university, they have a degree in English, and they’re baked, they’re good to go. But, what we realized with most of those folks is they have no innate ability to turn their writing abilities into generating revenue.

Gregg:

You know, on test we did came to mind, where we took … on the headline, we put in the word get. Get the skills needed to be a registered nurse, get the skills needed to be a cosmetologist, and we just added in the word get. We had, I think the average was a 28% lift, in the number of people that landed on the page, that would fill out a lead form. That’s crazy, don’t you think?

Tom:

Absolutely. It’s amazing, the small, little … How such a small, little tweak can have such an immense impact on what you’re doing.

Gregg:

Yeah, and what’s interesting to me also about testing is people who running testing don’t have to be gurus. Folks who listen to this podcast, if you have some marketing company that comes up to you with some big home run idea, like on Mad Men or something, Tom and I encourage you to run as fast as you can, away from those people, because they’re not telling you the truth. There are just too many external factors that could ruin … We’ve turned around so many ruined marketing plans with schools, because they’ve just gone whole hog in on big ideas.

Gregg:

What we want to do Tom, eh, is if someone has a cool idea, is to test it, and create a little safety net underneath, and some boundaries. So that, if the test goes sideways, the school doesn’t get hammered financially. You’ve turned around a couple of schools in that regard, haven’t you?

Tom:

Yeah. It’s funny, it reminds me. There’s actually a LinkedIn article today, that I read and I commented on, very interesting, on marketing. It was the title, “Which one are you, the tortoise or the hare?” The tortoise was talking about company’s marketing strategies where you’re plodding along slowly, more cautiously, 10% improvements, constantly testing. Or the hare, going for the big idea, as you said, trying to hit the big 10X home run too quickly.

Tom:

It is a combination of both. You have to come up with, hopefully, a big idea, or a niche product, or something that really makes you stand out. But, before you can just roll it out, and say it’s the greatest ever, you’ve got to split test it, you’ve got to implement it, maybe a little bit slower, with your clients. So, how you deal with the clients may be a little bit more of the split testing, process improvement, be the tortoise. How you deal internally might be, “Hey, here’s some big ideas, let’s test them out, see if they work.” And if they do, great, then we can slowly figure out how to roll those out.

Tom:

But, it really boils down to, “Here’s what we know will work, versus what we think will work.” That’s where your split testing person comes in. It can’t be just a whim that, “Hey, let’s do this, because it looks good.” Or, “We think it’s going to work.” Or, “This is great copy,” or whatever. Let’s test it out, and we implement only what we know will work, not what we think will work, after we test.

Gregg:

Exactly. Then, it becomes like packed in wisdom. You know, my wife has a health food store, and she takes all salad material, and she packs it into Mason jars. People take them home, and they undo the top of the Mason jar, and the salad just springs out. So, it’s kind of like you take all these little wins, you pack them in, and then what happens is, before you know it, you have a 20% life in enrollments. You’ve done it in such a way that you’ve not had to spend big money on ad agencies, or buy tons of media, or hire gurus, or new people. It’s an incremental, trial and error approach. What’s cool is anyone can do it.

Gregg:

What’s interesting is, that I get really excited about … Because my job is cross functional, through different departments, when there’s a problem or there’s something that we want to improve, ideas can come from absolutely everywhere. So, when you have a Split Test Manager, if you will, and that Split Test Manager communicates out to staff at large, and says, “Here’s a problem we want to solve on our marketing side. And if you guys have any ideas, just fire them your way.” The ideas for split testing, Tom, they can come from totally unrelated areas within a school. You want to expand on that?

Tom:

I think it comes down to having, and we’ve referred to it in the past, a culture of innovation. That’s encouraging people within your organization to come forward with ideas, in a safe space, where they at least felt they’re being heard, things can be tested, in all areas. And again, not just marketing, or sales, it could be in delivery, the product, the packaging, the HR functions, whatever, the finance functions. Any areas can be split tested.

Tom:

I’ll tell you, one of the … Not to talk about my kids again, but my youngest daughter’s a senior in high school, and she’s looking at a college, at going to college here. She’s trying to figure out what kind of a major she wants. I’ll tell you, one of the biggest majors that I’m seeing everywhere, and I think it’s in the top five for 2020 and beyond, right now, is data analytics. So, somebody who dives into the numbers, and is able to put those numbers into an easy format for the management to work through split testing, being able to grab the data, and analyze it, and find out where we’re leaking, or how the improvements can go.

Tom:

I think it’s this all working together somewhere, to create this culture of innovation and improvement within a company. And those companies that can consistently improve on what they’re doing, and tweak something, as well as have a good long-term vision, and a plan, but if they can continually improve, three, four, five percent here, two, three percent there, one percent here, all that adds up to a 20, 25 percent growth rate, on an annual basis. That’s a home run. It’s by having an entire organization, all working together.

Tom:

In our field, the school industry, I encourage instructors to present ideas, financial aid, student services, everybody needs to be constantly coming up with something new, that we can test and try out, with no fear of failure.

Gregg:

Yeah, no fear of failure, that’s the key. You get transactional managers who are driving expectations around results, yet not really giving people the tools to achieve excellence. So, if you have a transactional manager, there’s no way I’m going to go and offer up ideas around split testing. The reason is because, as you’ll point out in this afternoon’s client mastermind, most split tests fail.

Gregg:

So, as I was thinking about this just this second, here, you coach elite soccer, I’ve been in sports all my life, like you. I was thinking back, in college, I was on a rugby team that did not lose a game for two years. What we were well known for was the ball wound spin out to the wing, and rather than giving it up, or kicking it away, we’d hold onto it. Reform, it’s called a phase, and we would often have five or six phases. Literally, we would go backwards, the wrong way, down the field. It was totally counter intuitive, but we kept testing, and trying to push through the opposition. Usually, phase three, four, five, six, somewhere in there, they would breakdown, and then we’d go through, and we’d score a try.

Gregg:

I know in soccer, you coach at a high level of soccer, it’s the same thing, right? About not being afraid to go backwards. You want to just touch on that, for a sec?

Tom:

Absolutely. It’s the exact same principle that we were talking about, where sometimes you’ve got to go backwards to go forwards, and reformulate. If you look at, and you read The Art of War, or The Leadership Secrets of Attila the Hun, if you read those types of books, you constantly get this. They’re probing for a weakness somewhere, and sometimes you retreat to attack. The same thing is in business. Sometimes something works, we learn from it, and we try something else. But, you’re constantly trying and testing.

Tom:

Like I said, in war you might be probing the enemy lines with little attacks here, little attacks here, little attacks here, until you find the weakness. You fail a bunch of times, but if you find that one area that’s a weakness, that you can exploit, get your whole army behind enemy lines, and now you’ve come through victory. So, it’s not all just being able to smash your way through something, it’s test, fail, test, fail, test, fail, test, win. Then, potentially could turn into a huge win.

Gregg:

Yeah. Then, with the failures, the key is, folks on this podcast, is when you’re doing a test, you always put a safety net underneath the test, so that … Like Tom is saying here, is if the test fails, you just go fall into this little net, and you don’t hurt yourself.

Gregg:

Like in baseball, you might fail three-quarters of the time, and be successful at one-quarter of the time, and if that’s the case, you’re making millions of dollars. I mean, with our company, our success ration on split testing is somewhere around 20%. So, what that means is basically, if we do five tests, four of them fail, and one of them succeed. Then, our clients benefit.

Gregg:

But, if we were running in a typical, got to plow through and make a profit, and charge, charge, charge, we would be seen as a loser, a big bunch of losers here at Enrollment Resources, because 80% of the things we try, fail. But, if you fail with open eyes, and a safety net underneath, and you have a victory like putting the word get at the front of the headline, everybody, all the boats go up, all the clients results increase and improve.

Gregg:

Isn’t that weird, eh? It’s two ways, Tom, of looking at testing. One is testing for victory, one is failure. “Dummy, you failed 80% of the time.” That’s really, I think, what a lot of businesses bring, is don’t fail.

Tom:

Right, don’t be afraid to fail, is what we see out there. One of the reasons for that too, and we see it on the human side of things when we work with schools on admissions, counseling or something like that as well, too, is its outside of your comfort zone to try something new. Most people are very unwilling to go outside of their comfort zone, which is where true growth happens. “It’s safe in here, I don’t have to go out there, and I don’t want to try something new. I don’t want to try a new script.” Or, “I don’t want to try a new product, because I’m afraid of what it could do.” Or, “It could make me look bad, because I was advocating not doing it, and if it works, I’ll look bad.”

Tom:

It’s really a matter of having a mindset that you’re willing to try, to expand, and having that learning mindset. Which we’ve advocated here, at Enrollment Resources, for as long as I can remember, as one of the key traits of a top performer, is having a learning mindset, which means wanting to grow, learn, and try new things. You’ve got to have that learning mindset to be able to adequately take on split testing, and be proficient at it.

Gregg:

If an owner or a leader of a school, or a department head, are not willing to having intellectual curiosity themselves, and not willing to embrace failure, and trial and error, their employees are not going to do that for fear of losing their job. So really, the work we can do is to work with the leaders, to have them reframe, and reset their attitude in terms of embracing failure. Great companies do that, Apple comes to mind, Toyota, companies like that. Why not the school?

Tom:

Mm-hmm (affirmative), absolutely.

Gregg:

Any final words? I think we’re done on this topic, Tom.

Tom:

No. The only thing I want to say at the end, here, is you’ve got to be open, and willing to split test. You’ve got to put resources into it, and make it part of your culture. It’s amazing what can be done. Like I said, just from two little split tests … I hate to say the word little, because they take some time, and somebody has to come up with an innovation to begin with. But, two split tests can mean, as I’ll show today, hundreds of thousands of dollars a year making, again, that person … If that happens once a month, that’s $1.2 million a year, if you’re just making an extra $100,000 a month, through one little tweak split test. It’s amazing growth, and you’ve got to be willing to go forward.

Gregg:

Tom, that’s crazy, that’s crazy. It’s almost non-believable. I think that’s where people should just come and talk to us, and see the light. See the light of how split testing works, and can make your balance sheet really hum, and make your operational surpluses very surplus-y, and all kinds of great things. It empowers and excites the stakeholders within the school, to think of new ways. They become, in addition to their job, they become lab technicians, and they’re testing and trying things. It creates a little crackled of excitement, so many reasons.

Gregg:

I think, folks on this podcast, it would behoove not to spend some time with Tom, to go through our process called Scorecard, which is really a way to teach leaders how to split test, and use trial and error, in varying scenarios within the enrollment management path. Feel free to contact us.

Gregg:

Well Tom, good chatting. We’ll see you again soon.

Tom:

Sounds great, look forward to it.

Announcement:

This podcast is brought to you by Enrollment Resources, innovations in enrollment management. Learn more at www.EnrollmentResources.com.