I’m going to lay out the things I did and wanted to do in the first 30 days of my growth engineering job at Credit Karma. I work on acquisition: getting us more traffic and more signups. My engineering work contributes to our efforts across SEO, conversation rate optimization, UX, and page speed optimization.
The following areas are important to me:
1) I wanted to gauge the amount of A/B testing and experimentation we've done across the key areas of the site. Neglected areas which are still in the conversion flow probably have easy conversion wins waiting for me.
2) I wanted to measure our copywriting skill and amount of copy testing, because as patio11⁂ points out, a good homepage headline can create a 20% site-wide lift.
3) I wanted to tap into the voice and exact words used by our members so that I can find good headlines and copy and understand their painful, expensive problems†. This gives me the background I need to read a page and know how well it connects, and what CTAs will resonate best.
4) I wanted to understand WTF people are actually doing on our site. What pages do they visit most. Where do they bounce. What pages are dead ends without a CTA. What pages load like molasses. What pages suck on mobile. What pages look like when logged out vs logged in. What personalization can be shown when logged out, to continue to achieve the goal of the page. What other proven conversion tactics can we steal from other parts of the site and easily reuse. Where does the “scent,” aka intent, of a conversion flow get confusing for members. Where do they put their trust in us the most, and how can we reassure them at these key moments or ask for less (sensitive) information. And so on.
And here’s how I’ve gone about figuring out the above.
This is difficult because our particular team is on the newer side and we don’t have a well organized log of completed experiments, their results, and what parts of the site they were run on. Maybe if you use optimizely this would be easier to figure out.
We have certain phrases that have certainly undergone testing in TV ads. I have noticed that we have very few tests for headlines or button copy. I sense we have a lot of wins waiting for us here :) I suspect that our paid revenue marketing teams have some great copy in their Facebook and Google ads, but I’ve yet to request access so I can read what they’re running.
The best way to do this is with an Amy Hoy※ Sales Safari. Because I haven’t bought the course (yet) and I was feeling lazy, I just signed up for f5bot.com and set up keyword searches for Credit Karma and the names of our competitors to see what people are saying about us on Reddit/HN. You can also use littlebirdie for this. I also asked member support to give me access to any places where they post feedback from our members (for us that is the #all-the-feels channel and the #bad-iOS-reviews / #bad-android-reviews channels).
Of course I want more than that, but I haven’t been super focused on conversion copywriting and sales writing lately, so it hasn’t been the highest priority. I wish I had more good words from our members because we have a lot of different services and I feel I don’t understand people well enough yet (and I suspect my coworkers are in a similar boat).
To accomplish this I did the following:
This is very helpful because behavior flows tell you what pages someone visits before and after a page, as well as the dropoff rate. So you can get a high level feel for what people are doing.
I put all of these printouts on one of our large whiteboard walls. While giving people a tour of the wall and discussing it with coworkers, we found a lot of improvements to be made. We still have tests to run to see whether the improvements create lift. There are some that will create lifts with 100% certainty, such as adding a call to action at the end of a page that lacks one :)
I have yet to make a more exhaustive list of tests to run, and prioritize them. But I'm sure we'll get to it eventually.
I'd highly recommend this one for any growth team, because it gives you all shared context and lets you really SEE your site again. Looking at your site on a screen just shuts down your brain. Also, you likely don't frequently visit many of the top pages and flows that drive your business.
Someone on our team made a spreadsheet where you can plug in an incremental lift to any of the following metrics and it will spit out a revenue lift. I'd highly recommend you make one of these for yourself because it makes back-of-the-napkin impact sizing MUCH easier.
Here's what we can easily convert to revenue:
Good luck out there, may your tests win!
Growth Engineer at Credit Karma & consultant. Past clients include Aconex, Triplebyte, Neo, Brown Computer Science Department, Voxer, Cloudera, and the Veteran's Benefits Administration.