April 9, 2020
A LeadGen Company, a PhD Data Analyst & 200M Data Points Uncovered a Goldmine | AWasia 2016

A LeadGen Company, a PhD Data Analyst & 200M Data Points Uncovered a Goldmine | AWasia 2016


Alright, thanks. Thank you all for
coming to this presentation. Two and a half years ago,
I made a pretty bold move. I was doing affiliate marketing for quite some
time already and one of the biggest verticals I was running, and still am
running to this day, is sweepstakes and if you look at sweepstakes as a
business model, that’s like a company, at the end of the day they collect data and
after that they find clients to sell the data to. So as an affiliate, I was
thinking okay, I’m running big numbers right now, why not cut out the middleman
and with the middleman in this case, I don’t mean the network, but I actually
meant cutting out the advertiser. So what happened is I partnered up with three
others, also experienced in the industry already and we decided to just
start this venture, and now we’re two and a half years later and the lead
generation company is active in eight different countries, we have over 40
employees, we have three offices in different cities in the world and I can
tell it’s quite a success, so far. After that, we hired two people, we were active
in different countries, we did another very interesting thing, in my opinion, we
decided to hire a PhD in cosmic-ray data scientist, and this was a friend of a
friend of our CEO and at first, we were thinking, okay the link with online
marketing is pretty far off but at the same time, he knows how to work with data
and these guys, you can just hand them any data set you want and they will find
whatever you want them to find. So the beginning of this year actually, we saw
his first value. He was working in a company for over a year, but the
beginning of this year what happened is all of a sudden our clients told us you
have bad quality. So first thing we did was like, hey can’t be true, we have high
quality, we have good validation, we know how to work with the traffic, we
haven’t had these complaints before. So the first thing we did was like look at
our traffic sources and be like, it’s your fault or it’s your fault
or it’s your fault, and after we couldn’t really find
anything, we started looking internally. It should have been all the way around
obviously, but we did it this way and what we found out is that 16% of all the
leads we were selling to our clients we’re underage. So this was like, wow
what’s going on here, this is not how it’s supposed to work and this is
where Stefan, our data analyst actually proved his first value because this is
the point where we said like, okay please find out what’s going on, please
find out how we can fix this especially. Now the funny part comes, of that
16%, 73% of all the leads had as a date of birth, the
same date as the lead would sign up. So I can better show it with an example of
how it actually worked. Here you can see one of our sweepstakes. At the left hand
side, it’s just a form, where it’s not filled out yet and I took for granted
that if someone sees they have to fill in like the gender, they have to fill in
the first name, last name, email that it would be pretty pretty common sense
that if you see day, month year, that you also have to fill in your date of birth
there, but it did it happen. What actually happens was that people or well 73% of
the underage people, filled in the same date as when they registered on our
sweepstake, and this was like a moment first, where we were like wow,
what is going on. The fix was really easy. The fix was literally this. As soon as
someone would enter the date of today or something that doesn’t really make sense,
we will just show them an alert saying like, hey are you sure you’ve filled this
in correctly? And the result of this was that as opposed to 73% of all
underage leads signing over the same day, it went back to 0% so all
of a sudden, our clients were happy again, we didn’t have to block out any traffic
sources or whatsoever, and we fixed the problem. Einstein made a
very interesting quote years and years ago obviously. He made a quote where he said,
you know, humans aren’t that smart and I’m not trying to say that he is
right, but what I am trying to say is that in this specific scenario, I took it
for granted so much that if you see a day, month, year that people would know
what would happen, but actually I was the one who was a little bit stupid, in this
case because I made an assumption and this is actually what I want to
talk to you about today. I want to talk to you about how as an affiliate, it’s
really easy to make small assumptions or big assumptions just because you use
your common sense and what you should do as an affiliate is actually use data.
It’s pretty pretty obvious, everyone knows it, you can read it on STM and you
can read it everywhere, but still, some things you take for granted so much that
you think like, testing is not even needed. So the four things I want to talk
to you about today are shown on the screen right now and I’m actually going
to prove them wrong by using the data we have from staff and our data analyst and
you’ll be surprised by the results. So first one is that we were under the
assumption that all the leads that were coming from pop traffic as a traffic
source was bad quality, and why did I think it was bad quality because there
is this general assumption about where pop traffic is coming from.
I was running dating about seven to eight years back, on source like AtCache
and WeGotMedia and from my experience back then, they were mostly being shown
on torrent websites, on like watch free movies online or video games or
something similar like that, but what actually happened is that torrent
sites, they became mainstream and how did they become mainstream because there was
a lot of media outages about, for example, the torrent websites or people
actually start thinking, hey these torrent sites, I can
actually use them and it’s not just the tech-savvy persons who know about
internet that use them nowadays, but it’s also just a general population. What I
want to show you next is data we gathered from three different countries,
in three different part of the world. So I’m going to show you the
demographics of Brazil, UK, and Australia. I was very surprised when I got this
data from Stefan, hope you will be too. So Australia first. In Australia, 73%
of all our leads were actually over 26 years-old, 73% over 26 years old.
So what about the UK? UK you would think, okay people are more
known with the concept of pops, but it got even better. In the UK it was
77% of all the leads that was over the age of 26. Now you can
think okay, Brazil. Brazil is a way younger country in terms of internet
users and all that. In Brazil I was blown away by this fact Brazil, it was a total
of 81% of all the leads that were over the age of 26. It’s really hard to
say, like how this is possible. Of course, because I don’t have insights in
everything and our affiliates are also not willing to share everything for
example, but there is some common sense that you can use here, which
actually is that seven or eight years ago there wasn’t there weren’t any mobile
pops, etc. and so what really changed a lot about the entire pop inventory
available on pop as traffic source, it’s they’re coming off from mobile. Either way,
point one, where we were standardly thinking like, okay leads
from pops is bad quality, it’s been proven wrong. Next point, is that as an affiliate,
I was always taught right from the beginning, you have to convert the
user as fast as possible. If you can leave away some information, which you
think won’t hurt the conversion rate, then please leave it away, because we as
affiliates in general, we don’t care about quality, really. We care about
getting the click to lead ratio or connect to sell ratio as high as
possible, and the advertiser has to care about the quality part that’s their part.
Now, before I get into details about this one, I want to show you a
little bit from how the business model of sweepstakes actually works. You have
three big steps. First step is you have the form, where people fill in their
information. It’s also the same form where I showed earlier, where people put
in the wrong date of birth, and after that the affiliate pixel fires. So at
that moment, if you would be running sweepstakes that’s when you get paid, but
at that moment, as a sweepstake company, you didn’t earn money yet
because what happens after that is that the user gets put in a survey, in a
questionnaire that’s a middle part and in this questionnaire, we ask anywhere
between 10 to 85 questions depending on the country and it’s our clients who
can give the questions in there. So you have to think about it, that’s like there
is an electricity company who wants to get new customers, so they would approach
us and be like, hey we’re looking for leads who are interested in saving up to
20% on their electricity bill, for example. So then you would have a
question in the middle saying would you like to save 20% on your next
electricity bill? Yes or no? The moment a person clicks yes, that’s the moment
when the leads actually gets sold to the client, so that’s the middle
part. Then at the end, when all the questions are done, someone gets sent to
an exit offer that’s not really relevant for us, but it’s really about
the main part about the survey, where the money is made
for us. So with this affiliate mindset, what we were thinking is we have to keep
the questions as short as possible, like please don’t try to take too much time
away from the user because you might lose them that way.
Now after Stephan joined the company, he also dove into this kind of data and
looked at how many people actually end or go through the entire funnel. The
result was quite surprising because 81% of all the users who filled in the
initial form on step one would stay until the end. And by the end, I mean
the moment they answer the last question and they go to the exit offer. What’s
also really interesting is that on average, people take 13 seconds per
question that is being asked in a survey. I was highly surprised by this.
This is specifically for Australia, we have an average of 64 questions in the entire
survey, which actually means that from start to finish, people take an
average of 14 minutes to complete the entire sweepstake. Now how this helped us
big time is that before we, for example, they don’t want to show logos, we didn’t
want to make very extended questions, we wanted to keep it as short as possible
kinda, but this really changed our own mindset because we were thinking okay,
people are willing to read if they are highly engaged with what they can get. In
this case, with sweepstakes it would be something like win an iPhone, so they
really want to win that iPhone so they’re also willing to put in the
effort to actually read all the questions and go through the path as
good as possible. In our case,
it actually means that we don’t have to convert a lead as fast as possible, how I
would apply it to any other campaign is that again, for this one, it’s an
assumption that’s being made like you have to convert it as fast as possible,
but did you actually ever try it with a longer pre-lander or a longer offer page? Point number three, I guess this was
also a mistake made by me again, where I was thinking like, okay how the
general life of someone looks is, you get up, you have breakfast, you go to
shower and then you spent the next eight or nine hours at work. After that, you
come home, you have dinner, and then your free time starts. That’s the moment
when people would actually convert, that’s what I was thinking where
the conversion rate would be the highest, but is that really true? The next graph
I’m going to show you, is a graph of all the data combined from all the different
countries we are alive in, and it shows when a conversion rate is the highest, at
what point of today, and it looks like this. It’s actually really surprising to
me because it means that right around lunchtime for sweepstakes,
right around lunchtime, people are most engaged to actually sign up to the
sweepstake. Okay, so we got this data, we were thinking, okay maybe it has to do
with like one specific country or something where the conversion rate
would be higher and it would kind of distort the data we have here. So
then we took the data from all the different countries and there is
actually only one exception where it’s not true, and that’s the UK.
Don’t ask me why. UK it’s at the highest, the
conversion rate is actually right before people leave work, but still it’s if
you think about this, like how many times have you been thinking for example, okay
I have to launch my campaign at a good time of the day so that the initial
conversion rate will be highest and I’ll have like a good idea if the offer works
or not. What I would be doing before is I would always launch it around 7:00 p.m.
because then people are actually at home, they’re willing to convert, but this
data actually shows that it would be best to launch it early in the morning.
So again for us, point proven wrong. Last point and at the same time, this was also
the most interesting one, in my opinion and that’s where we were always thinking
that women would be higher quality quality than men, and the reason why I
was thinking it is that when I was running sweepstakes directly to our
competitors right now, they would always tell me, please only send
traffic from women who are 25 plus, because they are the highest
quality. So I was saying, okay I’ll take it for granted, why try to invent the wheel twice
when we start this company, so we would also be focusing on women opposed
to men and we could reason it with ourselves
because we were thinking like okay, women like discounts, they like to sign up for
offers, for voucher campaigns, they like to win stuff, while men are only
interested in gadgets and electronics and they don’t have patience.
That’s what we were thinking. This part is again important in order
for you to understand what I’m going to say next. So as I told you before,
as a sweepstakes company, you make the money in the middle part in a survey and how we actually calculate the value of a user is with a
term called RPT, its revenue per transaction. So what it basically means
is that let’s say you have ten questions in your survey and for each question, you
would get paid 10 cents if it’s answered positively and someone answers
five questions out of the ten positively, it would mean the RPT for us
would be 50 cents. Now so for us in order to look at quality, we actually
look at the amount of questions that are answered positively and at the end of
the day, that means that we’re looking at what our RPT is. What
we found out at first is that men actually convert 1.5 times more positive
than women overall, in every single country this data was like, I
didn’t understand it. I would think it was more equal or that it would answer a
different type of questions or anything but it didn’t okay. So what about ROI,
for obvious reasons I can’t show you our exact numbers but I’m going to
take a baseline, meaning that I will show you an RPT of $1 in
every country and then I’ll show you the equivalent how men would be looking
percentage wise. So first one, Australia. Difference isn’t that much,
it’s only 4% the sale is 4%. Next one UK, for every $1 we make for a
woman, we make $1.08 for men, 8% difference. The one where I was again
most surprised by is Brazil. In Brazil, we have 30%, we earn 30% more
for men, than we do from women. Another very interesting fact is that we were
thinking that women would take more time to actually read the questions and to
actually engage more with the entire survey in the beginning. The graph I’m
showing you right now is showing the average time it takes per gender to
complete one of the questions in the path and again, also for this one you
can see that men are actually performing better than women. So the patience part is
also not true. In our case, at least point being that we actually made a really big
mistake in the beginning, where we were just assuming that women would be
converting better than men showed you a lot of points right now and a lot of
data and of course, it doesn’t necessarily apply to everything but the
point I’m trying to make is that you always need to need to be thinking, you
always need to think smart, you need to think bold. What we did by hiring our
cosmic-ray data analysts is something that was against what a lot of other
opinions were of other people because they were saying like hey, online marketing
what’s the connection with them? Why would you give him such a
responsible role, such an important key factor in your company to someone with
zero experience? We took the dive, ended up really well. So what I’m trying to say
and this is kind of cliche but it is true, think outside of the box. Don’t
think there is a box, do something different.
Next thing I want to show you or tell you actually is to really cut out your
emotions. The screenshot you see here is from a movie called Moneyball and in
that movie, it’s based on a true story, it’s about how baseball changed, how it
used to work is where you would have scouts going to different games and look
at different people and they would pick people based on how their swing is or
just looking at them and and not actually fact-based or anything.
So what happens at some point is that there was a team in the US
and this team they didn’t have much money at all. So the very expensive
players at the biggest teams, they would be too expensive for them. Then there was
one analyst who started using the database of all the baseball players
around and he would just make decisions on hiring players based on the actual
statistics. In other words, they didn’t care how people looked, they
only cared about actual results. For us affiliates, it’s really
easy to make assumptions nowadays and you are not willing to test, but at the
end of the day, what I’m trying to actually say is at any point, even if you
think something is so common, is so true, keep testing. That’s it. Thank you. Awesome, thank you very much for that.
I really, really learned a lot. You probably know that I’m a closet nerd,
I used to be an actuary previously, so data was my whole life and it’s really
cool to see that people are bringing it into performance marketing, to affiliate
marketing. So we’ve got a few minutes for questions. I had one off the bat just to
get it rolling. Go for it. At what point, what volume of
data do you need to make it worthwhile to employ somebody to start analysing it
right? Because there’s some kind of tipping point, where now you’ve got
enough and you kind of assume that this person is going to more than pay for
themselves and give you some valuable insights. Was there a clear point where
you waited for enough data or you felt that you had enough in order to make it
worthwhile? It was more the other way around where we were thinking we
were tracking everything by the way, so every single click that was happening,
every event that was actually happening in the sweeps, we were
tracking. So we were storing everything in the database, but we weren’t doing
anything with it and then there comes a point where you’re like okay, it’s really
cool to have all this data, we have database tables of gigabytes, full
of points that were was not being analysed. So it’s I guess it came at the
right time as well, like we weren’t actively looking for someone like that.
We had it in the back of our head that we at some point needed to do with,
but this was I guess a case of being at the right time, being at the right spot,
at the right time. Right, good timing. So you’d already had
built up this data, it was sitting there. Yeah. And then the opportunity came along
to have somebody qualified analyse it and you grabbed it with both hands. Yeah.
Yeah very cool. What was the biggest insight that came from
the analysis that proved a previous assumption correct? So you
shared with us a ton of stuff that you know you’d assumed, like man versus woman
and the data proved it wrong. Was there something that you’d assumed
before that was kind of traditional knowledge that the data did prove correct? Yeah, quite a lot as well, because how it
works nowadays is that Stefan is proactively looking for weird
things by himself. How we had to do it at the beginning was we had to
give him points which we were thinking that they were
untrue right, but then I guess we were thinking the right way because they were
actually true. Got you, so most of the traditional assumptions in marketing. It’s a
lot of like one of the points, I was really curious about actually and we
spend a lot of time trying to figure something out that would be really cool
to show here as well, I was really under assumption for
example that the terms and conditions of sweepstakes would be read quite a lot
and it turned out as like everyone thinks about it, that people just don’t
read. They don’t read it. So that was a shame but. Very cool. Okay, so I’d like to
open it up to the audience questions. There are a couple mics scattered around
if anyone has any questions, Alex would love to answer them, I’m sure. Well there may not be as many nerds in the
audience as we think. I’m sorry? Maybe they’re not as many nerds in the
audience as we think. Data guys. Well, here’s another one from me. You mentioned
that it took about 14 minutes for them to complete the question. Yeah. Yeah, have
you guys have a thought of shortening it or would that reduce the value of the
data? Was that a discussion that ever came up? Was that even an option? Yeah it
was before because at some point, you’re thinking, okay we have so
many questions right now and we were so much under the assumption that the more
questions we would have, the more people would drop out. Exactly. That’s what we
were thinking, like okay at some point there must be a tipping point where people
are like, I can’t do this anymore, screw your iPhone, I’ll go for something
else, but it didn’t happen. Actually people just keep going. What
really did make a difference as well is that, and my thing at least, is
that right from the start, we had a little progress bar at the top. So people
knew how far they were off of actually completing it. We never tested it to
remove it because it makes so much sense to have it there, but I think it does
make a difference. Gotcha, so you definitely tested using less questions
but you felt the drop off didn’t justify the reduction in data that was collected?
I’m sorry? So you tested it having fewer questions but it just wasn’t worth it,
long story short. Yeah. Because the same amount of people completed it. Exactly, so
then at that point it doesn’t make sense to have the option to earn less revenue
if you know up front that you will earn more it all the way around. Does that
make sense? Yeah okay, so a technical question not to
do with the data analysis I guess. How do you deliver the leads to the people that
are buying them? Is there a platform that you use kind of a white label solution
or did you have to do something manually? We started off with Cake.
Actually Cake has a sub-platform which makes this possible.
I hope there’s no one from Cake in the audience, but Cake proved to be the wrong
platform for us, to say it politely. So then we started developing our own
platform. Okay. And this, I guess this is also the main strength of our company
right now that we have a very custom solution with very much validation and
there why we are able to deliver high quality. Very cool, okay. So last call, if
there are any questions from the audience? Hi Alex, I was just
wondering how difficult it was for you to find all the buyers for all the leads
in so many different markets. How? How, yeah.
How involved was the process especially like if you have Brazil and
and AU and all these other different geos. Was it hard? Were there marketplaces
you were able to tap into or did you have like connections already from when you were doing sweeps as an affiliate or? As I told you before, I started
the company with all our partners as well and actually our CEO is probably
the best sales person I know. So he knew how to get the clients in and I knew how
the traffic worked. So that was kind of like a good match. Honestly speaking, I’m
not involved in a sales process at all. So I don’t know how they hunt them, but
my assumption is LinkedIn and then you know just searching online for people
who have newsletters because they will be interested to buy data.
Cool, thanks. Welcome. Yeah. I was just wondering and I’ve done
a bit of sweepstakes, what’s up with the other competitors I guess. Like with the
25 plus women. They never test it or how do you explain that? So if I understand
your question correctly it’s? Like your assumption that women 25 plus are
performing better compared to men, which is still going on today right. I think
it’s still the case, yeah I just know that whenever I was sending younger
people I would get cut off from the other sweepstakes companies out there. So
basically they just don’t test the difference between men and women or
seems to me that that’s the case. Okay. Cool.
Welcome. Another question that Hugh just mentioned
was, he’s tried to hire somebody like this to analyse his data and it was
pretty challenging. In your case, I think you briefly mentioned it was a friend of
a friend. Yeah. Yeah, do you have any suggestions or did you try to hire
through a more traditional channel previously? I think honestly
speaking, if I would want to hire someone for this role, I would also look into
someone who has like at least knowledge of online marketing, but if you really
think about it, and I also talked to Zeno about this right before this speech,
it’s all about analysing. So if you know how to analyse data, it doesn’t matter
in what field you specialise, you should be able to work with any data set.
If I would have to do it again right now, I would just look up
what kind of studies are there that involve like statistics, and
and data analysis, and make your pool a lot broader them then just visioning
someone who already has marketing experience. Gotcha. Yeah because there are
a big pool of people, big data is a big upcoming market. Yeah. Something that is
very important though is you always have to keep it challenging for them. So
for us, it was really challenging I guess or for settlement was, because we
actually had like huge amount of data and no one else ever analysed it
before. So he was the first one who could also really work with it, which is like a
challenge for those kind of people and we were very happy with it.
How did you have to deliver the data to him, was he pretty flexible in
the format that he took it or did it take a lot of technical work, database
manipulation. He has like his own workflow,
so we pretty much gave him access to the database and he took it from there.
Awesome. That’s what’s amazing. Yeah.

Leave a Reply

Your email address will not be published. Required fields are marked *