Phonesandplans.com.au is a popular mobile price comparison site for Australians. It was launched 2 years ago and saw a steady increase in traffic up until mid last year. Then in April this year the site was hit by Google’s infamous Penguin update and lost 90% organic traffic. This is the story of how it got there and the long road back. I’m one of the co-founders and will take you through our journey.

Background

In May last year we decided to get serious about SEO and hire an agency. Thus far the site had good on-page SEO, but external links while not many were mostly organic. As we didnt have the time or the ability to do this ourselves, we decide to take the agency route. We were determined to find someone who fit within our limited budget yet practiced ethical, white-hat SEO. This was surprisingly difficult, but in the end we went with a local company we connected with at CeBIT 2011. They seemed knowledgeable, understood our market and most importantly, they matched our values.

Right. First couple of months as expected  there was no change in traffic. But from November we started seeing big jumps in numbers to the point that phonesandplans.com.au was the top result for our main keywords in December. We were delighted.

At this point, unknowingly, we had made our first mistake – we did not ask for details of the agency’s link building activity.

Then in Feb we had our first taste of the caprices of Google with the Panda update. Website traffic dropped by 10-15%. We discussed with our agency about what could the reason be but things quickly went pear shaped. They had no understanding of what had happened, why it had happened and what needed to be done to fix it. While the traffic impact wasn’t much, we realized that they were not the ‘experts’ they made out to be and we parted ways. This was probably the smartest thing we had done up until this point.

Then, the day after ANZAC day in April, Google Analytics showed organic traffic drop by 90% over the previous day. We had been Penguinned!

So what is Penguin?

It is a change to the Google algorithm to weed out sites that have been over optimized to show up higher in search results – too many inorganic links, no variations in backlink anchor text or any other form of a spammy link profile. (Wikipedia)

What happened

First step was to understand what had happened.  Phonesandplans is not a spam site. It is a labour of love built over two and a half years with the clear purpose of helping people. How did it come to be stuck with this spammy lot? Was it possible to recover from Penguin or did it make sense to start from scratch with a new domain? It was time to Google Google (metaGoogle?).

After much research and help from some SEO gurus, we finally had a plan of action.

Step 1: Analyse Link Profile

There are free tools available to get a view of a  website’s backlink profile – two good ones are Majestic SEO and Open Site Explorer.  Infact Majestic SEO is fantastic. They provide free reports for webmasters that let you see your detailed backlink profile. We discovered that there were over 3500 external links going to www.phonesandplans.com.au and of those 2,700 were for one single phrase ‘mobile phone plan’! No wonder Google came down hard on us.

The report also provided a list of top referring domains, almost all of which were unknown to us (at that point). These results were backed by Open Site Explorer and a couple of other backlink checkers.

Clearly the SEO agency had no idea what they were doing, but were doing a lot of it.

We contacted the agency with these reports and asked to get rid of the spammy links. As expected, they washed their hands off the whole thing saying there was no need to worry and this was just a passing blip (been a month at this point). But if we did want the links removed, we should pay them a stack of cash. So that was the end of that.

Step 2: (Try to) Remove spammy links

Next we developed a process to manually contact each website with a request to take down the offending links.

We put together an Excel spreadsheet (sample here) with a list of all spammy links from Majestic SEO reports and then set about finding the owner details. Most  times running a whois test produced site owner details. The SEOmoz  extension for Chrome was very useful. In cases where Whois was inconclusive, we tried to find a contact form or details of a parent company.  Once a working process was established, we were able to outsource this task to our data entry team.

Then we contacted each one of the sites with a request to take down our links. Here is the email we sent out.

Dear Webmaster [contact name if available]

I’m writing on behalf of the website phonesandplans.com.au regarding a link on your website [RefDomain] on the page [URL] with the anchor tag ‘Mobile Phone Plan’ linking back to our site.

Our site was penalised by Google for inorganic links as part of their recent Penguin update and we are requesting all links to be taken down in order to recover our rankings.

We would really appreciate it if you could remove the link from your website at [URL]. Thank you for your cooperation.

Best regards,
SEO Team
www.phonesandplans.com.au

We followed up the email with a review a week later to see if the link was removed. If yes, we sent a thank you note. If not then a second follow up email. All the progress was diligently recorded in the spreadsheet so we knew exactly what was going on at any point.

But at the end of 4 weeks the email campaign, thorough as it was, produced exactly one removed link.

Step 3: Reconsideration request

At this point we decided to put the link removal campaign aside and submit a reconsideration request to Google via this form requesting the penalty to be removed.

The form is basically one big text box where you put in your story. This is what we covered:

1. About PhonesandPlans and what it stands for – i.e. providing unbiased, comprehensive mobile plan comparisons.

2. Details of the SEO Agency and our experience with them as described here.

3. We also had a go at them for creating this situation – where SEO has become so important yet complicated that website owners have little choice but to hire agencies or spend ages learning to do it all themselves. It isn’t enough to create a good product, you have to spend either time or money, and in most cases both, doing SEO; or be prepared to lose out to competitors with deeper pockets going for it all guns blazing.

We didn’t talk about the link removal efforts in this request as they hadn’t really worked. And kept the tinfoil hat off by not mentioning Adwords even once (that this whole update is just a ploy to push people into spending more with Google).

The request went out in June and then we just forgot about it not really expecting a response. By then we had moved on to Plan B which was a brand new website under a new domain and a new vertical – OzCompare is an extension to our price comparison platform for mobile broadband plans, with the potential to grow into other verticals in time.

Around mid July I happened to log into Webmaster Tools and there it was – a response from Google Quality Team. They had read our submission and come back with a standard copy paste message (can Google be dobbed in for unoriginal content?) declared that there were still too many inorganic links. Basically they were saying ‘Naivety is really no excuse for spammy link building. We’ll have you sweat it out some more’.

Second reconsideration request

So we ran another pass on the backlink removal campaign and this time with a lot more success. Many of the sites in the dodgy blog networks had been taken down and we were able to get quite a few more responses with our third attempt.

We wrote back again this time with more information to establish the credibility of the site and the team behind it. This time the reconsideration included:

1. Information about the founders and our backstories including LinkedIn profiles and published writeups establishing us as experts in our fields.

2. How the site adds value – by saving people time and money. Specifically talked about the Mobile Deal Finder that helps people make apples-to-apples comparisons between confusingly priced mobile phone plans and took us 6 months to develop.

3. Supporting site stats from Analytics – Low bounce rate (30%), High repeat visitor rate, independent site reviews.

4. More information on the SEO agency. We even included excerpts from the emails with their stance on the situation and also provided their contact details and LinkedIn profiles.

5. Described Link Removal efforts which until that point had been in excel.

They got back within a week, this time by email, asking for more details about the link removal efforts. We moved the details into a Google spreadsheet and shared the spreadsheet link directly.

Success! Sort of.

On the 28 of August – 4 months after the first hit, Google emailed back that the reconsideration request had been approved and the penalty was going to be removed.

An excerpt from that email -

At this time, you can consider this reconsideration request to be
complete, but please be aware of the quality of links to your site moving forward. There are still some inorganic links to your site, and we’ve taken action on those, but we’ve kept our actions extremely granular, focusing on those remaining links. The manual action applies to inorganic links that we detected during the time that we saw unnatural linking activity. This action is intended to discount inorganic links while preserving organic recommendations. It is not currently affecting new links to your site, so going forward, organic links may be seen as a positive signal for your site in ranking.

Phew! But it goes on to say

Please be aware that it can take time for changes to be reflected in our search results, so you may not immediately see the effects of our adjusted actions.

That bit certainly seems to be true as we haven’t seen much of a change in search engine traffic yet. Still down to about 10% of January numbers. Other sites have reported it can take upto 3 months for the sites to fully recover.

Lessons:

1. The initial Panda hit was most likely due to the template nature of our site. We have a page for each mobile phone in the market listing all available plans. The plans are updated every 3-4 weeks, but if you are Google only looking at the plan names you probably wont see much changing between pages.

2. If you hire an SEO, don’t go in blind. Keep asking for reports of what they are doing and how they are doing it. Paid links are an absolute no-no.

3. Prepare well before applying for reconsideration. Google want to see you sweat.

4. More is more. Provide as much information as possible on why your site is not spammy. Establish personal credibility, there isn’t much they dont know about you anyway.

5. Finally, be patient and hang in there. But have a plan B in your backpocket.

So what’s next for PhonesandPlans?

We’ve decided to stay away from the whole Google dance as things change all the time and it is impossible to keep up. Google ranking algorithm is far from perfect and they will keep doing what they need to do to improve it. Once again an update to the Penguin update is expected anytime now.

Our strength lies in creating great products and SEO is a huge time sucking distraction. Instead we are investing resources on tools and projects to drive greater engagement with our users and expect that to be a lot more sustainable in the long term. So stay tuned.

Did you find this write up useful? Have a Penguin story to share? Talk to us in the comments.