The 'apples' page on BBC Food

Hi, I'm Oli Bartlett and I was the product manager for BBC Food during the rebuild in 2009-10. This post is a follow-on to Duncan's SEO post to provide a little more context and detail on how we tried to maintain our audience reach during the re-launch.

In the BBC we often see temporary drops in audience reach after a major re-working of a website. In situations where a website is given such a significant overhaul that its structure and page URLs change, one major factor in this drop in traffic is the removal of the old URLs from the site.

Put simply, if you remove the pages and those pages were getting views, then you no longer get the views.

Once search engines discover that their indexed URLs are no longer valid (i.e. they receive a 4xx http response code), they will remove those pages from their indexes. In order to maintain the traffic from search engines it's important to put in place a good http response strategy for those URLs. For example, where content has been moved rather than deleted, use a 301 response code to redirect to the new location.

bbc.co.uk/food

Part of the problem with the old BBC Food website was that there was too much content duplicated in different forms across the website - for example we often had two or more pages displaying the same recipe - which is really bad for users and SEO.

Additionally, a lot of the content was due a refresh in the context of the new product goals - finding recipes and food from your favourite BBC programmes. This led to the decision to cull around 2000 pages from the old website - these included recipes whose rights had expired, duplicate recipes, and articles and other content which simply didn't fit with the objectives for the new product.

Three Kinds of Deleted Page

Each deleted page, or group of deleted pages, required a different approach to http responses:

1. Expired recipes: 410 - Gone. We present a message explaining the situation regarding rights to BBC recipes, and giving links to similar recipes (where recipe rights have expired we still know the detail of the original recipe so can link to similar recipes - ie for the same dish, by the same chef, using the same ingredients etc.).
2. Duplicate recipes: 301- Moved permanently. One of the duplicate recipes was kept, the other was deleted from the system and a 301 redirect put in place from the deleted recipe to the new canonical one.
3. Consolidated articles: We created 'food' pages (e.g. bbc.co.uk/food/apple) which acted as canonical resources containing the typical editorial content found in our old food articles (ie how to prepare, choose, store etc.). Each deleted article was 301 redirected to the most relevant food page, and in the case of articles about diets, occasions, cuisines etc. we had appropriate canonical pages for each.

Sometimes, 404 is the right answer

We tried to minimise the number of URLs that returned "404 Not Found" but invariably there were some which were removed and had no suitable alternative.

In this case it was considered to be better to return a 404 than to redirect to the food homepage.

Simply redirecting all removed pages to the homepage breaks the web. For example, if someone has posted a link to a page that subsequently gets removed, by putting a redirect to the homepage you give the impression (to users and search bots) that the post was about the BBC Food homepage.

Additionally, if a recipe search result links you to the BBC Food homepage, that's not helpful and you're less likely to click on a BBC link next time. We'd prefer those links to be removed from search engine indexes so people don't have that experience.

For the few weeks following the relaunch of BBC Food we were getting significant numbers of 404/410s reported on the site, but these were expected.

As the invalid page links were removed from search indexes, very quickly these errors tailed off.

The new pages were soon indexed and after a brief dip, our audience figures were back and rising healthily. We didn't completely avoid the post-launch dip, but it was predictable and reversible and so much easier to stomach.

Oliver Bartlett is Product Manager, Olympic Data, 2012

The Olympic Data Services Team at the Broadcast Centre in White City

Hi, I'm Oli Bartlett and for the last 15 months or so I've been the Product Manager for the BBC's Olympic Data services.

My team have built the systems which provide all of the London 2012 data to the BBC Sport Olympic website, mobile applications, IPTV applications and other BBC websites showing Olympics content.

We provide three main functions:

• The Dynamic Semantic Publishing platform (DSP). This is the framework for creating over 10,000 athlete pages, plus a page per event, discipline, country and venue.
• A service to receive, process and store data from the Olympic Broadcasters' Data Feed (BDF).
• A stats service, the Olympic Data API providing all of the sports data: Schedules, starting lineups, results, records, medal tables and video logging data.

The BBC, like other broadcasters around the world, gets all of its Olympics data via a feed delivered over the internet and the majority of its Olympics video from the host broadcaster, Olympic Broadcasting Services (OBS). The London 2012 Olympics are the first where a combined video and results data feed (the BDF) has been made available. If you'll excuse the abbreviations: The BDF provides Olympic Data Feed (ODF) results and statistics from Locog, plus video data from OBS. In practical terms, this means that not only do we know the live results for every moment of every event, but we also know which video streams were showing the action at that time.

The BBC subscribes to the BDF and we receive it via our data provider Deltatre Media Ltd. The raw BDF is huge and contains a load more data than the BBC needs. Additionally the feed is stateful, so requires a relatively complex receiving system which knows how to apply updates as they arrive. Deltatre transforms the complex and stateful feed into a simpler, stateless feed which provides the BBC with just the data needed to power the online products. For example, a BDF update for a football match may say that Team GB have scored a goal. If you want to display a scoreboard, this information is of little use without knowing the previous state (score) in the match. The updates we get from Deltatre don't require the BBC to hold state for each unit, so the football update would instead give the current score: GBR 1 - ITA 0. This has allowed the BBC to simplify its processing of the data as we are not required to build the complex business logic required to manage stateful updates of statistics across the 10,000 units of competition. Instead, we can focus on getting the data out to the audience as quickly and reliably as possible.

So what does this mean for you?

Well, if you've used the interactive video player you'll probably have seen the Extras button in the bottom right of the player. One of the options here is the athletes panel. This shows you a list of the athletes competing in the event(s) you're watching. Additionally you'll see a picture of the athlete, a link to their page on BBC Sport, and their vital statistics - height, weight, date of birth. If you drill down further you'll see statistics from the event in which they're currently competing. This all works off our Olympic Data API which can serve up the latest results and statistics from any time during any of the 2,500 hours of video during the games. This means that when you rewind a live diving event, or watch a football match from yesterday, the athlete data and match stats stay relevant to what you're watching. Alex talks more about the interactive video player in his blog post.

In addition to the sports data, the Olympic Data API provides chapter markers and the Olympics Live data to the interactive video player. These features are primarily driven by the video logging messages we receive from OBS. Each time something interesting happens on one of the multitude of video feeds, the event will be logged by a team of people in the International Broadcast Centre, in Media City and at venues around the country. Whether it be a record being broken, the start of a new quarter in a basketball match, or a foul in a football match, we will get a notification. We take these notifications and, according to a set of business rules, create chapter points or Olympics Live alerts. As the games get underway we can monitor the frequency and accuracy of the chapter points and live alerts, and can change these rules dynamically if required. Additionally we have ways to manually add chapter points or live alerts for those moments the logging hasn't captured.

That pretty much summarises the Olympic Data services. Dave Rogers, the technical lead on this project, will be writing a follow-up post which will go into more detail around the architectural and testing challenges we've come up against, and the team's development approach.

Until then, enjoy the games!

Oliver Bartlett is Product Manager, Olympic Data, 2012