• Stars
    star
    110
  • Rank 316,770 (Top 7 %)
  • Language
    HTML
  • Created over 10 years ago
  • Updated over 7 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A demo of how to allow social media crawlers (Facebook, Twitter, Pinterest etc) to read your AngularJS app

AngularJS Social Demo

This is a project which aims to illustrate how to enable rich social media sharing of your AngularJS app. This was created to accompany a talk on the same subject I am giving at the Vienna AngularJS meetup on 30th June 2014

A working demo of the following implementation can be found here, and all the code is contained in this repo.

Background

When sharing a link on social media website such as Facebook and Twitter, the site will send a crawler to the URL being shared in order to scrape data from it to make the link richer. The data scraped typically includes a title, description, image, and potentially much more. A common way to convey this information is via the Open Graph Protocol (used by Facebook, Pinterest, Google+). Briefly, to use it one would include special meta tags in the header of your HTML document:

<head>
    <meta property="og:title" content="My Page" />
    <meta property="og:description" content="A description of my page." />
    <meta property="og:image" content="http://www.mysite.com/images/my_lovely_face.jpg" />
    <!-- etc. -->
</head>

These meta tags are then read by the crawler and the contents are used to generate a richer sharing model (e.g. by including the image and description in the Facebook timeline)

The problem with AngularJS apps is that the entire view is built by JavaScript on the client, but (as of this writing at least), the various social media crawlers do not execute JavaScript on the target URL, and as a result they will only see the raw HTML template without any content. To the crawler, your template would look something like this:

<head>
    <meta property="og:title" content="{{ page.title }}" />
    <meta property="og:description" content="{{ page.description }}" />
    <meta property="og:image" content="{{ page.image }}" />
    <!-- etc. -->
</head>

Solution

The solution is basically to use some kind of server-side user-agent detection to pick up whenever a social media crawler arrives, and then instead of showing it the plain AngularJS template file, redirect it to a server-generated page that will contain the desired meta tags, all filled with the correct information.

What we will need

  1. A web server capable of URL rewriting. In this case, we are using Apache and the mod_rewrite module.
  2. A server-side language to generate our crawler-friendly pages. In this case I will use PHP.
  3. The Angular app must be using "html5mode" its URLs. This is because the # portion of a URL does not get sent to the server, so makes server-side redirection based on the Angular page impossible. For more information, see this StackOverflow answer.

Following is a write-up of how to set things up assuming the above technologies are being used.

Configure Apache

We will need three specific Apache modules enabled: mod_rewrite, mod_proxy and mod_proxy_http (installation of these modules will vary depending on your OS/Apache version, but is beyond the scope of this article). We will come back to the use of these modules shortly.

Set up the server-side script

Next we need to make the script that will handle the requests from the social media crawlers. Let's assume that our AngularJS app gets its data from an API. In this example, we are getting album information from the endpoint api/{id}. We can re-use this same API in our server-side script and use the data to build, on the server, our HTML page including all the social media meta tags, and output this HTML to the crawler.

An simplified PHP implementation follows:

$SITE_ROOT = "http://www.mysite.com/";

$jsonData = getData($SITE_ROOT);
makePage($jsonData, $SITE_ROOT);


function getData($siteRoot) {
    $id = ctype_digit($_GET['id']) ? $_GET['id'] : 1;
    $rawData = file_get_contents($siteRoot.'api/'.$id);
    return json_decode($rawData);
}

function makePage($data, $siteRoot) {
    ?>
    <!DOCTYPE html>
    <html>
    <head>
        <meta property="og:title" content="<?php echo $data->title; ?>" />
        <meta property="og:description" content="<?php echo $data->description; ?>" />
        <meta property="og:image" content="<?php echo $data->image; ?>" />
        <!-- etc. -->
    </head>
    <body>
        <p><?php echo $data->description; ?></p>
        <img src="<?php echo $imageUrl; ?>">
    </body>
    </html>
<?php
}

The output of this script can be tested by visiting it directly in the browser. In the example, that would be http://www.michaelbromley.co.uk/experiments/angular-social-demo/server/static-page.php?id=1

Redirect crawlers to the server-side script

Now that we have our server-side script up an running, we just need to set up the redirection. This technique requires the use of the three Apache modules mentioned earlier, and is done with an .htaccess file containing the following rule:

<ifModule mod_rewrite.c>
 RewriteEngine On

# allow social media crawlers to work by redirecting them to a server-rendered static version on the page
RewriteCond %{HTTP_USER_AGENT} (facebookexternalhit/[0-9]|Twitterbot|Pinterest|Google.*snippet)
RewriteRule album/(\d*)$ http://www.michaelbromley.co.uk/experiments/angular-social-demo/server/static-page.php?id=$1 [P]

</ifModule>

The RewriteCond link looks at the user agent string to see if it matches the following expression. The specific strings used in this expression are based on the known user agents of the various social media crawlers (at the time of this writing):

  • Facebook: facebookexternalhit/1.1 (+http(s)://www.facebook.com/externalhit_uatext.php)
  • Twitter: Twitterbot/{version}
  • Pinterest: Pinterest/0.1 +http://pinterest.com/
  • Google Plus: Google (+https://developers.google.com/+/web/snippet/)
  • Google Structured Data Testing tool: Google-StructuredDataTestingTool; +http://www.google.com/webmasters/tools/richsnippets

The [P] flag causes Apache to perform a remap using mod_proxy and mod_proxy_http, rather than a regular redirect. If a 301 redirect is used, Facebook for example will link to the "static-page.php" URL rather than the original URL.

If you are using NGINX

# NGINX location block
# this will PROXY-ly map (not 301 redirect) all the crawling requests from facebook, twitter bots... to the server that hosts
# the template for sharing. On the template, we will diplay only the meta tags in the head (See the template file in this same gist)

# This block goes first
location /cars {
  if ($http_user_agent ~* "facebookexternalhit/[0-9]|Twitterbot|Pinterest|Google.*snippet") {
    rewrite ^/cars/(.*) /cars/$1/share_for_proxy_passing break;
    proxy_pass http://the-server-that-hosts-the-template-share-for-proxy-passing.com;
  }
  rewrite ^/cars/(.*)$ /#/cars/$1 last;
}

# Then this
location / {
  try_files $uri /index.html;
}

Test it out

Now that everything is set up, it's time to test out whether it actually works as expected. All the social media sites we have mentioned so far offer some kind of validation tool that will give you an idea of what your URL will look like when shared:

Also, Fidder is a great tool for testing out this kind of thing, since you can manually set the user agent and then inspect the response from the server.

More Repositories

1

angularUtils

A place where I will collect useful re-usable Angular components that I make
JavaScript
1,999
star
2

ngx-pagination

Pagination for Angular
TypeScript
1,213
star
3

soundcloud-visualizer

Audio visualization with web audio API, canvas & the SoundCloud API
HTML
449
star
4

chromata

A generative art tool.
JavaScript
376
star
5

angular-es6

An experiment in using ES6 features with AngularJS 1.x
JavaScript
370
star
6

angular-wordpress-seed

A bare-bones AngularJS blog app designed to work with the Wordpress JSON REST API.
JavaScript
310
star
7

horizonal

Turn your static HTML document into a programmable slide show
JavaScript
205
star
8

css-space-shooter

An old-school arcade-style 3D shoot-em-up rendered entirely with CSS 3D transforms
JavaScript
176
star
9

skqw

JavaScript Audio Visualizer
TypeScript
120
star
10

angularUtils-pagination

Simple pagination for AngularJS
JavaScript
42
star
11

drawACatApp

A canvas line drawing experiment
JavaScript
29
star
12

heavenly-glory

A kung-fu movie starring you. Web RTC & Web Audio experiment.
JavaScript
23
star
13

css-experiments

Some experiments involving CSS
CSS
14
star
14

michaelbromley.co.uk

The source code for my personal website and blog
JavaScript
13
star
15

headlesscommerceplatforms

A feature comparison of various headless e-commerce solutions
TypeScript
10
star
16

shader-playground

Some snippets of code for fragment shaders
GLSL
9
star
17

cmpg

A super simple component generator for Angular 2
JavaScript
9
star
18

breezejs-odata4-adapter

An experimental adapter to allow BreezeJS to work with an OData v4 server.
JavaScript
7
star
19

skqw-library

Visualization library for SKQW
JavaScript
5
star
20

angularUtils-uiBreadcrumbs

Automatic breadcrumbs directive for use with angular-ui-router
JavaScript
3
star
21

vendure-plugin-example

Handlebars
3
star
22

canvas_experiments

Simple examples of HTML canvas
JavaScript
3
star
23

vendure-demo

TypeScript
2
star
24

angular-issue-18478

Reproduction of Angular issue 18478
TypeScript
1
star
25

pad

A new way to work with text.
TypeScript
1
star
26

angularUtils-disqus

A directive for including Disqus comments in an AngularJS app
JavaScript
1
star
27

gulp-es6-seed

A minimal seed to kick start an ES6 project
JavaScript
1
star