DISQUS

Christina Gleason · 9 months ago

I completely agree. I've seen suggestions that Twitter could slap nofollow on all new Twitter accounts until they reach some benchmark that "proves" they aren't spammers, so that legitimate users can pass link juice through their tweets and the "mosey along" crowd gets none of it.

Christina Gleason's last blog post..Do Bloggers Even Know They?re Breaking Google's Rules?

Jerry Okorie · 9 months ago

Awesome post - Andy. You've touched a very vital topic which Google should seriously look into or we'll start looking for a better search engine. You quoted "One way would be to either remove the “nofollow” tag from Twitter and let Google spider the links being shared. It has no need to be afraid of the spam–if after 10+ years Google can’t tell a spam site from a legitimate one, then maybe we need a better search engine!"

Thanks Dude..

Will Critchlow · 9 months ago

Did you see my post over at SEOmoz on a similar subject recently? I agree with you that not only is it necessary for Google to follow nofo links but I believe they most likely already are.

If you had a 'nopassjuice' tag (I hope they don't do that) why would you then ever use nofollow? As long as 'nopassjuice' innoculated you against linking to spam (which it would have to- otherwise twitter would be penalized) then there's be little use for nofollow would there?

Will Critchlow's last blog post..What makes designing for the web different to designing for Print?

Andy Beal · 9 months ago

@Christina & Jerry, thanks for your input!

@Will - great minds think alike! Although I suspect there are many of us that share the same sentiment. I agree about the "nopassjuice" tag. Google could simply amend nofollow. However, specifically for sites like Twitter, they might want to have a different option.

symbian · 9 months ago

I don't think that delays in indexing of new fresh content by Google is an issue. New story most likely will appear in top of Digg stories and then in all major *trusted* news sources. And all this job will be done by real people including checking and filtering. This is the basis Google rely on.

William's last blog post..Free Skype VoIP client in Nokia phones

Jacques @ SEO Tools South Afri · 9 months ago

A very pertinent and thought provoking post, Andy! The no-follow attribute is there to prevent link juice being passed, but it is actually proving to be an issue with regards to access to information being blocked now, so there definitely has to be a rethink on this issue!

Andrew Girdwood · 9 months ago

What? No.

"Nofollow" is an odd name. Google does follow those links. It simply doesn't let PageRank (trust & authority) pass through that link.

If you're argument is that Google can't discover new URLs from a nofollow link on Twitter then that's wrong. Equally, Google can "follow" the "nofollow" links on Twitter and be better aware of which blog/news site may have published a story first and that's a very important battle to win.

Andrew Girdwood's last blog post..Facebook to launch new homepage

Dave · 9 months ago

Google easily checks the "importance" of topics happening on nofollow links (lots of tweets clustered together in time on the same topic, for example) and can apply that importance to followed information on the same topic. But this is an imperfect solution.

Creating walled gardens around content (nofollow) to keep spammers/black hats out is also an imperfect solution.

You suggest "perhaps we need a better search engine." Of course we do. We always will. Perfection is not attainable, but improvement is. And Google is constantly improving, and most of us who really care about search don't rely solely on Google (Twitter search, anyone?)

Dave's last blog post..Share your blog posts everywhere

Andy Beal · 9 months ago

@Andrew is that your opinion--which you're welcome to--or are you making a statement of fact?

According to Google: http://www.google.com/support/webmasters/bin/an...

"it lets you easily instruct robots not to crawl a specific link." and "How does Google handle nofollowed links? We don't follow them. "

If you've seen a Google statement that contradicts the above, I'd love to see it. :-)

Jaan Kanellis · 9 months ago

"create yet another tag that instructs the search engines to follow, but to not pass on any “Google Juice” from the site doing the linking"

So the page doesnt get any internal link juice from the domain it resides on, but it is more than welcome to get outside link juice from other sites? Damn well that would be interesting and I am not entirely sure it would work the way Google does things.

Can Google strip the vlaue of pages internal link juice with just one tag? Matt?

Jaan Kanellis's last blog post..Make A Movie For Great Rankings

Andy Beal · 9 months ago

@Jaan - I was thinking more along the lines of a tag that Twitter could use to say "follow this link, index it, but we're not vouching for it ourselves." Some speculate that Google does that now with "nofollow" but I've not seen any evidence of it.

Jaan Kanellis · 9 months ago

OK so no link juice in the URL, then you wont be seeing many Twitter posts ranking Google. Probably as many as you see now.

Jaan Kanellis's last blog post..Make A Movie For Great Rankings

Andy Beal · 9 months ago

@Jaan - that's fine. The last thing I won't is to find Twitter posts in my search results, but if a Twitter user finds a link to a page it could get indexed much faster than just waiting for Google's spider to find it outside of Twitter.

Andrew Girdwood · 9 months ago

@Andy

Ah yes. It's Yahoo that may discover URLs.

However, I have seen a very convincing study which seemed to show Google used twitter links to discover content. The problems? Firstly the twitter account RSS had been subscribed to and it was suggested that aided content discover in itself. Secondly the URL shortner used provided another route to the content by listing the links elsewhere.

Andrew Girdwood's last blog post..Loic Le Meur and the IdeasProject.com

David Leonhardt · 9 months ago

I'm sort of with Andrew Girdwood. I have seen too many links indexed that have NoFollow attributes attached. To be sure, I see them more in Yahoo, but I assume that's just because I check Yahoo for backlinks much, much more frequently. I saw several links reported just this morning that were in fact NoFollow. I actually wrote a highly speculative post on that a couple months ago at http://www.seo-writer.com/blog/2009/01/14/how-i...

So I would take it as fact that the search engines can and do follow NoFollow links, and they can choose to treat them in whatever way they wish. I would consider it highly speculative to suggest what they do with those links other than index the.

David Leonhardt's last blog post..The Bookmarketer adds Tipd, Plugim, Plime

Barry Welford · 9 months ago

An excellent topic, Andy. I am completely against the nofollow attribute, which is being used for purposes not covered by its original creation. Even then it was not a good idea.

My two alternative suggestions are most unlikely to be acceptable to the big G.
a) Forget the whole 'nofollow' bit, or
b) Accept that the big G does not cover the total Web but only that durable part of it where URLs persist. Leave the NOW web to others.

The latter is surely best for Google if it wishes to maximize its business objectives, however I think it is still going after that impossible dream.

Barry Welford's last blog post..The Best (and Worst) In Customer Service

Chelle · 9 months ago

I guess the nofollow was supposedly a way to "hurt" anyone selling links on their sites...but really it seems kind of stupid when you think about it. I'm sick of everybody saying Google is so much better yadda yadda...when you really use google a lot though you realize it's pretty terrible. Gmail is tough to navigate, adsense is great but again, awful admin interface, and I usually find myself switching between multiple search engines when I'm looking for something because they don't always show up in Google for what I really need!

Chelle's last blog post..Ten Warning Signs of Alcoholism

saurav · 9 months ago

What if Google altered the definition of 'no follow' attribute wherein it followed the link for crawling and indexing purposes but did not pass any page rank or use this signal for ranking purposes. Though it would solve the issue in question (inability to index new content) but give rise to another - search engine ranking because its algo till date depends heavily on inbound links as ranking signals.

But then we have 'do follow' (set by default) which allows it to do both, crawl and index and use the link (a vote) for ranking purposes. So webmasters can choose the link attribute accordingly. If this be the case, we would not need another attribute for our links.

As far as spam is concerned, and as you said in your post, if Google can't effectively detect spam then we something better.

Great post Andy. It is always a privilege to read your posts.

Mike Quale · 9 months ago

I have to disagree with a lot of what's in this post. Your right in a couple of areas, like google and twitter should compliment each other. I am pretty sure they do. I a large number of back links from twitter. However the nofollow is much more valuable to twitter being there instead of removing it. If it were to be removed spam would follow.... "ooo looky a new nofollow site to affiliate market too Whooopie!!"

If you think google cannot spider millions of 140 byte hunks of test daily from a site around the block. Well your wrong sorry. Twitter has a feature in their API which is referred to as the "firehouse". All tweets in the public timeline are available to anyone that can handle that data. Google can still follow a "nofollow" link. I have seen them in my own webmaster tools from notorious nofollow craigslist posts. There is no juice, but that content is still linked, indexed, and spidered according to relevance. Twitter is an extremely relevant site based on Google's algorithm, hell they have a page rank of 9 for the love of all that's holy how much more relevant do you think you can get.

Google is not a real time search engine. Why do they need to be on top of the news again? it's a reference resource not CNN. Google aims to be an accurate reference guide to many, not just a traffic generating utility as many web business' would like them to be. If your on the web to make money, you better have good value in what you offer. As google gets smarter and faster more junk will be weeded out. SEO will be less prevalent as personal search is getting closer to reality.

Sematic web, not yet but getting closer.

PerfectMOney Share Idea · 9 months ago

Until Now.I doubt whether Google can determined which site that is spam or not ,however ,combining twitter and Google search for better result that is another problem,both side has there own ego
"NOT to follow the other one"this what make no profit can be taken by common people who look for specific information

Ari Herzog · 9 months ago

Looking at this entire debate from a different angle, keep in mind that ex-Google manager Katie Jacobs Stanton is President Obama's new director of citizen participation; and Twitter CEO Ev Williams was at the White House to help spur the economy.

I'm going to guess more's going on behind the scenes than we think.

Ari Herzog's last blog post..Use Blog Photos with Creative Commons

Jimi St Pierre · 9 months ago

Given that most of what is said on Twitter is personal, ephemeral, fun but not meant for worldwide consumption, I cannot really see why we are surpised that Google wants to protect us from a proliferation of such ephemera. The fact that marketers want to hijack Twitter to promote business is just another example of how a good socially cohesive idea is being used for business promotion. Why not keep it as a network of followers and followed? I for one would not like to see my search results full of spammy links - we spent enough time filtering that from our email life!

Pilgrim Writer · 9 months ago

Great post Andy: deserves to be climbing fast on Sphinn! :)

I too am against no-follow and think your suggestion makes a lot of sense. I don't feel, however, that Google will go in that direction. Sad but I'm overall disappointed in Google lately. It's become too big for its boots.

Lewis · 9 months ago

The reason Twitter first went nofollow was because Miss Google advised Twitter to. I remember the news some time ago (sorry, no details in my head and just about to run for a coffee) :-)

The nofollow was added to give the whole site more "welled up juice", so that of course twitter would then become more valuable.

At the flick of a switch Twitter could release all tags... so probably best to create an actual debate ON Twitter WITH Twitter... maybe =0>

Andy Beal · 9 months ago

@Mike says "Google is not a real time search engine. Why do they need to be on top of the news again? it’s a reference resource not CNN."

I beg to differ. Google News and Google Blog Search is evidence that Google wants to provide breaking news.

@David - Yahoo is a completely different beast. :-)

Anya · 9 months ago

Are you sure Google doesn't crawl links that have the nofollow attribute. I find that hard to believe really.

Anya's last blog post..I need some advice on what could be a life changing decision.

Andrew Girdwood · 9 months ago

@anya

I've seen lots of evidence to suggest that they do - but its really hard to test. For example, one twitter test I saw forgot that the URL shortner made the URLs available elsewhere which Google could discover.

SEJ has some pretty clear cut responses from the search engines on this matter: http://www.searchenginejournal.com/how-google-y...

In short; Google doens't follow nofollow. It's actually better for webmasters that they don't. It allows you to funnel your spiders a little better.

Andrew Girdwood's last blog post..Loic Le Meur and the IdeasProject.com

Skyper · 9 months ago

@Andrew, @Andy: Google *does* follow "nofollow" links, but apparantly does not count these incoming links when it ranks the site the links point to: http://gulati.info/2009/03/5-benefits-nofollow-...

This article may not be authoritative, but is in line with my observations...

Spidering and rankin are quite seperate things....


.S.

Andrew Girdwood · 9 months ago

@skyper

Hiya. Yahoo does. Ask does. Google does not. Really - it's in their own guidelines. This is even echoed by suggestions from various Google engineers that you should consider putting nofollow on unimportant links on your site in order to encourage the spiders to visit the more important pages.

Check out: http://www.google.com/support/webmasters/bin/an...
"How does Google handle nofollowed links?

We don't follow them. This means that Google does not transfer PageRank or anchor text across these links. Essentially, using nofollow causes us to drop the target links from our overall graph of the web. However, the target pages may still appear in our index if other sites link to them without using nofollow, or if the URLs are submitted to Google in a Sitemap. Also, it's important to note that other search engines may handle nofollow in slightly different ways."

I have seen a number of studies which seemed to show that Google did follow nofollow links. However, none of the studies were water tight as there were always other possible routes for engines to discover the content - for example, Google's Feedfetcher bot ignores robots.txt so if you provide an RSS reference to a page that's otherwise linked to with only nofollow links then Google may still become aware of the URL.

Andrew Girdwood's last blog post..Loic Le Meur and the IdeasProject.com

JMorris · 9 months ago

Andy,

Good post with many thoughts to consider. There were a couple that stood out for me...

if after 10+ years Google can’t tell a spam site from a legitimate one, then maybe we need a better search engine

There is the problem. Google cannot fully differentiate between quality content and spam. If it could, splogs would not reach first page ranking on SERPs with millions of results.

Hey, I'll admit it, I love Google in a lot of ways and I use it daily. It, IMHO, is still the best at providing relevant results. However, Google is not flawless and has made some big mistakes. The nofollow attribute is one of the tops IMHO.

create yet another tag that instructs the search engines to follow, but to not pass on any “Google Juice” from the site doing the linking

I really cannot believe you said that either. I "get" what you are saying, but why should a search engine be given Cart Blanche to pervert the HTML/XHTML specification by introducing yet another tag that will again FAIL. No matter what SEs do, spammers will be hot on their tails. Introducing more attributes and tags is not the solution. Better algorithms are the solution.

Good food for thought!

I agree the nofollow attribute needs removed on micro-blogs. Google is shooting itself in the foot by not indexing these links.

JMorris's last blog post..ImpressCMS Persistable Framework: Rapid Modules Development

gestroud · 9 months ago

I guess I need a point clarified before I can form a conclusion on the matter.

If the links were set to "follow," would they show up in Google's index as shortened urls (tinyurl, shorname, etc.), or would they links be translated into the actual page link?

Also, if they aren't restored to their original format, would it create duplicate content?

John McElhenney · 9 months ago

Andy,

What about all the Twitter____.com sites starting up daily. Seems that aggregators could get past the "nofollow" fairly easily. There are a lot of nice Twitter Link Gathering tools these days.

Are you concerned about Google not getting the entire story, or Twitter not getting all the love, or the hassle with which those of us who "work" not "game" the system will have to go through to clarify the truth. Cause I think ultimately that's what it's about, "truth."

There will always be gamers entering any system. Just look at the twitter-porn and twitter-spam starting to hit the fan these days. I find some poetic justice that the Auto-Follow and Auto-DM users are the ones getting pegged with the crap followers, but that's just me.

Rock on, and thanks for the information. I'll look to see if I can find someone doing the "follow-no-juice" method of link building and report back. Or you could let me know if I'm on the right track or wacked.

@jmacofearth

John McElhenney's last blog post..Microplaza for Twitter Groups and Tribes - Get'm Goin - Now Follow This!

Antoine Perdaens · 9 months ago

Andy,

May be this is a door wide open for a new kind of search engines. We already see people switching their default firefox search engine to twitter leaving google for occasional search.

We build a new search tool based on the hot links shared on twitter, we call it Tribal Seach and it is a new feature of our MicroPlaza tool, a link aggregator for twitter.

You can try to search for what's hot around search on twitter right now : http://microplaza.com/search?q=search , you will find interesting results.

@tlg

Tina Langely · 9 months ago

I guess I'm just too new at this to understand all the fuss about "follow" and "no follow." I'm going to have to learn more about it! Do you have a website you can recommend that teaches about this in easy to understand terms?

Tina Langely's last blog post..Broadcasting from your computer

Christoph · 9 months ago

A very naive article..

You're totally missing the technical point of view.
It's not as easy as simply enabling this, and there is no black/white distinction between all those things.

Fortunately it's easy to mislead people with this, as seen in the comments.

Skyper · 9 months ago

@Christoph, could you explain why we are "totally missing it"?

Also, if one *carefully* reads Google's statement above ( http://www.google.com/support/webmasters/bin/an... ) that "using nofollow causes us to drop the target links from our overall graph of the web", it becomes clear that crawling, graphing and ranking are seperate issues! Not sure what they *exactly* mean by "dropping target links", but you can only "drop" what has been crawled first :-) And Google does not mention any ranking ("link juice") in the post...

However, the "specific examples of nofollow usage" in Google post clearly shows exmaples of ranking, rather than crawling. Eg. "help keep your site from inadvertently passing PageRank to bad neighborhoods on the web", "comment spammers", "automatically or manually remove the nofollow attribute on links posted by members or users who have consistently made high-quality contributions", "prevent paid links from influencing search results" etc.

I think this makes it quite clear what Google does ... (of course they word their statements very carefully to protect their search results quality - which is their #1 competitive factor)

.S. (sorry for the long post)

Chris Tew · 9 months ago

I'm pretty certain that Google still follows nofollow links. They just don't pass PR/authority, but still allow spiders to follow & index, and also pass keyword relevancy.

It simply prevents a site gaining sitewide authority through nofollow links, so that only true editorial links can give a site authority. But nofollow links can still get things indexed and improve rankings as Google accounts for keyword relevancy.

This gives Google the best of both worlds.

Skyper · 9 months ago

@Chris,

Agree with you 100%. And this would be the sort of approach that Google woukd take ;-)

.S.

JackHumphrey · 9 months ago

Google not follow a link? I think their statement you quoted above is just a ruse to put off the spammers, which I'm sure it has to a degree. But I've always had serious doubt, to put it mildly, about them not following (literally) nofollow links.

jhon · 9 months ago

hi!Nice article I like it .I appreciate this and agreed this.

Neyne · 9 months ago

It is a mistake to make a statement that "google will do this" or "google will not do that". The closest to the truth you can get is by saying "in large number of cases google will do X".

I have had pages ranked for a keyword that appeared only in the anchor text of a single, nofollowed link. The story is more complicated than Google's laconic statement.

Neyne's last blog post..My Thoughts on Google’s Webmaster Tools 404 Report

Dana Lookadoo · 9 months ago

Great post and discussion!

Wish Google had a Community Manager who listened and joined the conversation by monitoring and responding to industry-leading blogs, especially discussions like this.

Their Help pages don't always demonstrate what we see spit out into the SERPs. What @Neyne said is often true: The closest to the truth you can get is by saying “in large number of cases Google will do X”.

Dana Lookadoo's last blog post..Hyphens vs. Underscores - Who Wins “Best Keyword Separator?”

Brent Nau · 9 months ago

Andy,

m.twitter.com - all DoFollow links
search.twitter.com - all DoFollow links
RSS feed - all DoFollow links

I think someone referenced it above, majority of third-party API sites use DoFollow links.

So I would say that Twitter is NOT a Black Hole like Wikipedia.

Brent Nau's last blog post..Twitoria - Find Inactive Twitter Friends

Jimmy · 9 months ago

Well, I think it's twitter who needs to remove the nofollow tags from their site, If they are trying to fight spam they should be using some other methods.
Great blog!

Pipe fittings · 9 months ago

If you’ve seen a Google statement that contradicts the above, I’d love to see it.

Marc · 9 months ago

GREAT FREAKING POST!!! I agree fully and wish i could have chimed in sooner. These freaking retards are destroying the fabric of the internet by screwing with links.

Marc's last blog post..Omar al-Bashir Defies Arrest Warrant Issued by International Criminal Court (ICC)

David Bayer · 9 months ago

Andy - interesting post but not technically accurate. GOOGLE'S SPIDERS DO FOLLOW THE NOFOLLOW LIKS ON THE PAGE AND WILL IDNEX THE CONTENT ON THE OTHER SIDE OF THE LINK. You are correct in noting the fact that the nofollow doesn't allow the passing of page rank.

Robyn McMaster · 9 months ago

Hi Andy, I like the way you engage readers by putting opposing views upfront in a way that engages readers. So I have named your blog for a Premio Dardos award. The The Premio Dardos is “bestowed for recognition of cultural, ethical, literary and personal values transmitted in the form of creative and original writing.”

Robyn McMaster's last blog post..Premio Dardos Awards

Joost | Internetguerillas · 9 months ago

I have some experience and it is true that pagerank isn't leaked... but other factors like relevancy do get passed on...

PChieng · 9 months ago

It would help out so much if Twitter removed the nofollow tags on links in both the profiles and through the twitterstreams. To help increase one's PR on their profile they would could look at the number of followers vs followees, as well as factor in the number of updates. As far as links in the updates, it makes sense that if a link is popular, people will retweet it and Google will see those links, making the page more relevant.

And if you look at this post about Google needing SEO, you will see that they need to read this post and follow your advice to "breaking down the walled gardens".

~e~ · 9 months ago

You made one critical mistake: Google will FOLLOW NOFOLLOW links in order to find the content and index. What it wont do is give that link credit for search engine rankings.

Helen Palmer · 9 months ago

Great post and discussion! I think that Google are going to have quite a task keeping up with all the twittering.

basmin · 8 months ago

confusing but very informative.
I agree with Tina Langely, this must seem to learn more about the nofollow and dofollow

basmin's last blog post..Tukang Nggame

Or Hillel · 8 months ago

You right! twitter need to place "nofollow" only for people that in the site less than 60 days, al the other without...

press · 7 months ago

It seems that google does follow these links but not sure if they are counted towards page rank. Certainly appear on webmaster tools.