March 28, 2020
Adding AMP to an e-commerce Workflow, with eBay (AMP Conf ’17)

Adding AMP to an e-commerce Workflow, with eBay (AMP Conf ’17)


[APPLAUSE] SENTHIL PADMANABHAN: OK, cool. Thank you, Marco. Hello, everyone. Welcome back. My name is Senthil. I’m a Software Engineer at eBay. And today, we are going
to talk about adding AMP to an e-commerce workflow. So basically, you
have an organization which has a development
workflow, a code base. Also, a release process. And you want to put
AMP into that mixture, and you want to do
it very seamlessly. That’s the essence of this talk. Although this talk is
focused towards e-commerce, a lot of points here apply to
web development in general. OK. OK. Before getting started, I want
to give some quick background that will set the context
for the rest of the talk. So at eBay, we are planning
to build a new product browse experience at the same time
that AMP was announced. So this was a new and a
better experience for users to browse our products. We didn’t start the development
yet, but the concept of AMP resonated a lot with us. And one of the main
reason is because a lot of traffic to these
new product pages are expected from
search engines. And we wanted to provide a
very good experience for them. If a user is
already within eBay, we know where they
are going to go next. We can optimize
the flows for them. But if they are coming from
external platforms, like search engines or social media, we
don’t have control over there. And that’s when we thought
that AMP makes a lot of sense. So what we did was along with
regular page development, we thought that we’ll
also give AMP a shot. So there would be
the primary page, which is the page that
we are going to develop. And along with it, we will
also have an AMP version. We also went ahead
and set ourselves some goals on how we want
to do this AMP development. Our goal number 1 was
minimum engineering overhead. So I already mentioned
there was a team that was ready to work on this
new product browse page. So just for doing
the AMP development, we will not be having
a new team to do it, or we’ll be getting
new engineers to this team to be doing it. It is the same team that has
to deliver the AMP version. And one of the main reasons for
this is at that point in time, we didn’t know whether
AMP would really work for e-commerce, because it
was targeted more towards news and publishing sites. And AMP links were not shown for
e-commerce sites in the search results. Our goal 2 was no deviation
in terms of user interface or experience. This was a strict requirement
from our product and design team. They already knew that AMP
had restrictions and they said to us that just because
of the restrictions, you cannot compromise on
user interface or experience. So this was goal number 2. And goal number 3 is
analytics should be intact. I’m going to talk a lot about
it in the upcoming slides, so you’ll hear more. So fast forward today, eBay
has close to 16 million AMP-based product browse pages. These are all broad queries. So you can go to
Google, type in a query, and then if it qualifies for an
AMP, you get that result here. It’s pretty fast. It’s pretty solid. And users use it on their
mobile devices a lot. Yeah. So this is a query for
Canon digital cameras. How did we achieve these goals? So let’s see. And that’s what the rest of the
slides are going to be about. So our first approach
towards AMP was this. At the end of the
day, AMP is nothing but a bunch of best practices
for building web pages. Yes, there is a lot of
technology behind it. There is this AMP cache. There is this pre-rendering
and plenty other stuff. But all that is possible
because AMP mandates these best practices while building pages. So what we thought
was, why don’t we incorporate these best
practices as a part of the regular
development cycle itself? Like not only for the
AMP version, but also for the regular version. And also, for pages across eBay. So this gives us a more
organic approach towards AMP rather than it being
a forcing function. And also, it makes
our other pages fast. We sort of had these
things everywhere in eBay, like it was scattered
across, these best practices. But this exercise helped us
to consolidate that list. I’m just highlighting
a few here. There were a lot more best
practices that we follow, but these are the ones that
we immediately put in place. In terms of CSS, we
made them size bound. So the CSS required for the
initial rendering of the page should be less than 50 KB. We avoided expensive CSS rules. I know this list keeps
changing, but at least we have a list now. We use only
GPU-accelerated animations to get the 60 frames per second. In terms of resources, we
statically size all resources. So the width and
height of an image is always sent
declaratively in the markup. We also used the
appropriate pre-hints, like prefetch, DNS-prefetch,
and stuff like that. The second thing that helped
us to build an AMP version was a principle that we follow
in eBay for quite some years, even before AMP came into play
in the lean and accessible principle. We call it the LA principle. And what does lean mean? Lean means that
you build and ship only the minimalistic
thing required for the initial
rendering of the page. This is a very common advice,
but it’s often overlooked, and people send a lot of
unnecessary bytes on the wire. But it also means to
us– there’s another rule that CRP, that the
Critical Rendering Path, should not depend on scripts. So again, for the initial view,
you only need HTML and CSS. You don’t need
JavaScript for that. And that’s what
this rule is about. But because of that, we
have this flexibility that the scripts,
the JavaScripts can be loaded in an async way. And apparently, AMP also
does the same thing. AMP loads the scripts
in an async way. So our page-loading
pattern already matches. This made it very easy to build
an AMP version of our page. Under lean, we also have
the component-based approach where we build atomic
encapsulated components. To us, a page is nothing
but a bunch of components. The components can be granular. The components can be
like composite, which is, again, granular components. But it’s just a
container of components, that’s what a page is about. And fortunately for
us, AMP also follows the same component-based
approach, where everything in AMP
is a bunch of components. It helps us in two things. One is reusability, which is
pretty obvious where we can use the components everywhere. But it also makes our
application become agnostic of AMP switches. Meaning, let’s say you
have an image component. The application will just
call the image component and the switch
between whether I have to use an AMP image
or an image will happen within the component. It’s encapsulated inside that. So the application logic
becomes very neat and elegant. So it’s all encapsulated
independent components. So this is the one thing
which helped us a lot. Now, let’s talk
about accessibility. At eBay, we take
accessibility very seriously. Even the smallest accessibility
issue is like a blocker for us. And we are very happy
to see even AMP follows a very similar approach. They give a lot of
importance to accessibility. And under accessibility,
we say that you always start with an accessible
markup instead of it being an afterthought. Again, a very common
advice often overlooked. People first, when they
get the requirements, they build the
markup, and then they start putting
accessibility into it. This creates a lot
of back and forth between JavaScript,
HTML, and CSS. But if you do this, I
think it’s a good solution. And AMP also does a
very similar thing. They have an accessible
markup when they begin. But we also do one more thing. We enforce accessibility
through CSS. A technique that we follow
in eBay where the styles get applied only when
the right [INAUDIBLE] accessibility
attributes are present. You can read more about
it in the link there. And AMP also does a
very similar thing. A lot of state
mutations/interactions work only if you have the
right accessibility attributes. And this made us
a natural fit when you start building our pages. We liked it so
much that AMP also is following a lot
of regular patterns that we have been following. The third thing that helped
us in building an AMP version is another principle
we have in eBay which is code with the platform. The platform here
being the browser. So over here, as you
all know, the browsers have evolved and become
smarter and smarter. So the concept here
is try to leverage as much stuff that comes by
default in the browser instead of relying on heavy abstractions
or complex libraries. Of course, you can use
small libraries, but not heavy abstractions. So under this
rule, for this page we decided that we
will not tie up ourself with any big frameworks. At least for this page. Because this page, if you
see, it’s more of browse page. As I told you, it’s just a page
where users come and interact. You don’t need a big
framework for this page because there is
no complex state mutations or any
complex interactions or transactions happening here. And since we did this,
our markup looks natural. Meaning it looks like HTML,
just in a declarative way. And since it looks
like HTML, it becomes very easy to replace
with AMP markup because AMP markup
also looks like HTML. It’s all about custom elements,
which is a declarative HTML API. This was a big
time-saver for us when we were building the AMP version. It was very easy for us
to go back and forth. Till know, we have been seeing
how to code your application. Next, comes the part, the
build and release part, where you got to deploy
an application to staging, pre-production, production,
and all those steps. This is where the minification,
bundling, transpiling, and all those stuffs come into play. So what we decided here
is that we should have a streamlined build process. Meaning that between
AMP and non-AMP, there should not be anything
drastically different in terms of a build process. We already have a
build process in place. Let’s use the same
build process. The switch of a button
thing should happen. And under that, we
decided that the tool should handle the difference. Meaning there should
not be any handholding or any manual
intervention needed. The tools that we have
been previously using should also be
available used here, which means that
the tools should take care of the difference
between AMP and non-AMP. For example, inline CSS versus
external CSS, no JavaScript versus bundled JavaScript. Using the AMP image tag
versus the image tag. These are just a handful
that I have put here. There are a lot of other
things that happen here. Fortunately for us,
our asset pipeline tool called lasso, which
is also open source, has conditional
dependencies, which are nothing but flags
based on runtime context. Usually in the build process,
we have build-time context. But these are flags
based on runtime context. That when a request comes
to us as web servers, we know whether this
request is an AMP request or a non-AMP request. And the optimizations that I
mentioned in the previous slide get applied dynamically. So you don’t have
to do anything here. It all happens [INAUDIBLE]. The link provides more
details about how we did that. So we did all these things. All these things lead
to less fork in code. By fork, I mean the
difference between– the fork between the AMP and non-AMP. And in fact, 85% of the code is
shared between AMP and non-AMP. This was one of our
initial thoughts that we should achieve this. And with this, our
goal 1 was achieved, which was minimum
engineering overhead. Now, let’s talk
about AMP components. So along with AMP runtime
that comes default with AMP, AMP also has this thing called
AMP extensions or components. And to our surprise, it’s a
pretty exhaustive list, even with all the restrictions. It is pretty exhaustive. If you see any common UI
pattern out there in the wild, you can see it in this
AMP component list. But more importantly
for us, it was sufficient to build a
fully-featured product browse page. Of course, you cannot
do a transaction here. You cannot do a checkout here. And that was not our
intention anyway. We wanted to build a
browse page [INAUDIBLE]. And this component list
was more than enough. As you can see, the difference
between our AMP view and our non-AMP view. It more or less looks the same. It also feels the same. You can interact with it. The behavior is very similar. The only difference
is there is a search box at the top in the non-AMP
view which is not there. And even that’s getting
added in the March release of our AMP version because that
component is now available. With this, our goal
2 was achieved, where the user
interface should not be changed between an AMP
and a non-AMP version. Now, let’s talk about analytics. I told that keeping analytics
intact is one of our goals. And let me explain why. Analytics is super-critical
for e-commerce. I know it’s critical
for everyone. But in terms of e-commerce,
a lot of our product roadmap, business decisions are
all based on the data that we gather from analytics. And it was particularly
important for this page because we needed a robust
and granular analytics system. Meaning we needed to
know how users interact with every module in the page. We need to know how
[INAUDIBLE] the tap. Whether they did swipes,
scroll, and all these things that we know. And also, it creates an
automatic feedback loop to improvise user experience. Let me explain that. So on the first day of
launch, for the watches query, the above the fold
had an events module. This is a module that we
thought would work out for people who query watches. It shows all the events that are
associated with luxury watches. We thought that this is what
the users would like when they query the watches module. But what happened in
reality is the users, as soon as they came, they
scrolled down the page and they went to the
deals module, which was below the fold. So obviously, our
assumption was wrong. But fortunately for us, all this
data, this interaction, swipes, scroll, everything goes
to our back-end system. And our machine
learning team uses this data to train our
module ranking system. So what happens automatically
on the second day is this deals module,
which was below the fold goes above the fold. And it all happens
automatically behind the scenes. There is no rollout,
nothing associated with it. And this can happen
within a day. This can happen
over multiple days based on the amount of
interactions that the users do. So this was possible in the
regular version of the AMP page we did because this
was one of the main things that we wanted to achieve. So how did we do it
with the AMP version? And that’s where AMP
analytics comes into play. It was already talked about
in the earlier sessions, too. AMP analytics gives a pretty
exhaustive tracking mechanism. So it is config-based. It can be page visibility,
horizontal scrolling, vertical scrolling,
a couple of stuff. But it was missing one thing. It was missing element
level overrides. Basically, associating
tracking data with individual
elements in the page. I already mentioned
that we needed a very granular tracking. We needed tracking at an
individual element level. And that was missing here. We could have done
it with configs, but that would have
not been manageable, because there would be configs
everywhere in the page and you cannot just keep using that. So as any other good open
source citizen, what we did is we filed [INAUDIBLE]. We [INAUDIBLE] and also
contributed towards it. And now you can add a
[INAUDIBLE] track attribute in any individual
element that you want, put all your
tracking data inside, and that gets automatically
sent to your analytic system. So with this, we
are able to achieve the same sort of
mechanism that we were doing with our regular
pages, even in our AMP pages. But analytics
doesn’t stop there. Another important thing
is session switching. So in the e-commerce world,
just a user looking at a product alone is not sufficient. I mean, we are happy they
are looking at the product, but we also want them
to go ahead, proceed, and do a transaction. And when they do
a transaction, we need to attribute the source to
where it originally came from. In most of the cases,
it will be within eBay. It can be from
third-party affiliates, or it can be from AMP. So we need to know
what the source is. And this is what
session stitching is. But the thing with
session stitching is it is very easy when
you’re on a single domain. For example, you’re on ebay.com. You control your domain. You have the cookie. You own the cookie. You control the flow. You know when to start the
session, when to update the session, and everything. But it becomes
complex when you are dealing with multiple domains. And as you would
have guessed, AMP has multiple domains
associated with it. So how did we achieve
this session stitching in an AMP [INAUDIBLE]? So I will walk through the
scenario on how we did that. So when the user first
comes to Google Search, they click on a link. The AMP view shows up and it
is hosted in [INAUDIBLE], which is not an ebay.com domain. What we did is we set up a page
visibility or page impression tracking on load of
the page, which gets [INAUDIBLE] when
the page is visible. And that call goes to an
eBay analytic service, which is hosted in [INAUDIBLE]. It also passes a page
ID with them, which is the page of the AMP view. This analytic service now
knows that this request came from an AMP view. And this creates a session
accordingly saying, oh, this session
created by AMP view. And sends back a
set cookie header in the response which has
the session details put in along with the domain, which
is ebay.com, with the expiry and a lot of other stuff. Now, when a user goes and
clicks on any of the link here, they go to a page,
the eBay view, which is hosted on ebay.com,
which is an eBay domain. And since this previous
cookie was set on ebay.com, that cookie gets traveled
to this eBay view. And this eBay view calls
the analytic service with the same
cookie information. And now, that
system knows clearly that this was a session
that originated from AMP. And then it can add more data
into the [INAUDIBLE] session and tie this page
view also into it. And it keeps continuing. Was this clear? So when the journey
ends, it’s pretty much people are aware that– I mean, our analytics is
aware that the journey originated from AMP. And what happens
in a repeat visit? So same thing
happens pretty much, but one difference is in the
first page impression call, a cookie also gets– the cookie
that was previously set also gets traveled to the
[INAUDIBLE] point, which is an eBay analytic service. And now, the analytic
service can be smart. It now knows it
is the same user, but they have come on
a different session, or they have come on
a different query. So they can do all
the mix and matches on how the session works and
set the cookie accordingly and all the remaining
things works as such. So this is how our session
stitching exercise worked. And with this, our goal
3 was achieved, which was analytics should be intact. Now, let’s talk
about challenges. We have been talking about
the good things till now. Of course, there
were some challenges. But most of the
challenges are now gone because as the
AMP project progressed, a lot of announcements
were made in the project that the challenges we
had earlier are now gone. One thing I highlighted
is the tracking challenge we had where we needed
a granular tracking. That is sort of fixed now. We needed an input box. We have an input box now. We needed a tab component
and that’s there now. I mean, one of the reasons
is because we were one of the early adapters of AMP. And at that point in
time, the component list was not as big as it is today. So today, we have a
pretty exhaustive list, as I mentioned. So you can pretty much
build everything there. Of course, you cannot
do a transaction. And when we get to
that time, then we’ll have some restrictions
or challenges. But for now, we are good. But there is one thing
that I want to highlight. This is common with many big
organization or infrastructure components. So infrastructure components
are those critical components that are built by
various teams across eBay and get included
automatically in your page. And they are really
mandatory components. So they have to be in your
page, like global header/footer, site speed we can
create, experimentation libraries, security module,
and things like this. These all get automatically
injected in your page. The application teams don’t
even have to code this. And the problem is all of them
have JavaScript in them, which makes them automatically
disqualified for putting them into an AMP version. So this was a big
challenge for us. So making them AMP-compliant
was much more trickier than we thought because
there were a lot of policy restrictions in the company. We got to work with various
teams on getting this result. So if you are doing this,
please consider this. This will result in scope
creep and complexity in your development cycle. So if you are a new organization
getting started with AMP, I think this is one of
the pieces of advice that we would give you that
we learned the hard way. There is also a caveat
in session stitching, which I mentioned earlier. Safari cookie policy does
not allow third-party read and write cookie for
first-time users. Meaning if a user has
never visited ebay.com and they go to the Google
search results, click on a link, and AMP view shows up, the page
impression calls gets fired. And in the response, I mentioned
there is a set cookie header. That cookie doesn’t get set. Which means when the user
clicks on an AMP page and they go to an eBay view,
we don’t have session data. We think that’s a new user
just coming from ebay.com. So this, obviously, resulted
in a lot of high bounce rate. Because we were
assuming initially that all our users came to
AMP and they just went away without even doing
a transaction, or without coming to eBay
and visiting our site, which is obviously
a false positive. And honestly, we don’t
have a good solution for it yet because it’s
a kooky security policy that’s [INAUDIBLE]. We have a partial
solution here, like what we do is we retrieve
the [INAUDIBLE] traffic from ampproject.org in the
same bounce rate interval, and then we subtract it
from the bounce rate. This sort of gives
us a solution, but you can’t always
rely on referer headers because refer headers
is not session. Because the same user
can go back and forth. It will be treated as a new user
if you just stick to referer. Because referer doesn’t have
any other context built into it. So that’s why we do not
recommend this fully, but this is what
we have in hand. And we are going
to work more on how to try to resolve this because
all our metrics are sort of skewed now to get that
exact information on how the transaction works. Coming to the last
part of the thing. So what’s next? So we are also planning
to build an AMP version for our core product pages. So currently, AMP is only for
a subset of our product pages. There is a huge other
list of product pages where the look and feel
is quite different. They are more narrow
queries, like when you drill down the queries. And so those are the pages
that we are looking to build. We are also looking into
progressive web AMPs. It’s pretty obvious. This looks like a
natural extension for us. Since we already have an
AMP, a progressive web AMP looks natural. This is something that
we are also looking to build sometime this year. But what is our ultimate goal? So what’s the target that
we are setting, right? So at eBay, we want to provide
a fast and frictionless commerce web experience irrespective
of three things. Where the user
starts the journey, the underlying operating system
or device, or the network conditions. And AMP is just one
step towards that. As you know, the web
has evolved a lot. And it gives us the
power to do these things. We just have to apply it
to the whole spectrum, starting from
Credential Management API to Payment Request Web API. I think if you apply it, we
will be there pretty soon. Thank you. [APPLAUSE] [MUSIC PLAYING]

1 thought on “Adding AMP to an e-commerce Workflow, with eBay (AMP Conf ’17)

  1. Google's aim is to make web developers develop web sites the way they
    want. So the Google search engine will discover the content easily show
    the google searchers and promte their business. They understand that
    their search algorithm is not successfull anymore. Instead of making
    their search algorithm understand the available techs they want you to
    design the way that their algorithm undestand. It seems that web sites
    will look like as it was in late 90s.

Leave a Reply

Your email address will not be published. Required fields are marked *