Prevent Googlebot Following Links in Gmail

If you run a mailing list or send automated emails with links, you may have noticed something strange: Googlebot is following links sent to Gmail subscribers, which can trigger unsubscribe actions or inflate download counts.

Why Common Solutions Don't Work

The typical approach—using robots.txt or adding rel="nofollow" to links—doesn't work for email links. These methods only apply to web pages, not email messages. When Gmail displays emails in the web interface, Googlebot can still crawl those links.

The Solution: User Agent Filtering

The best approach is to detect and handle Googlebot requests at the server level. Check the user agent string in incoming requests and filter out bot traffic before processing actions like unsubscribes or incrementing download counters.

When you detect the user agent string contains "Googlebot", you can:

• Return a simple success page without processing the action
• Log it as a bot visit instead of a real user action
• Return an HTTP 200 response without side effects

This way, Googlebot can crawl the link (keeping it happy), but your application doesn't treat it as a real user action.

For more details, see Google's official documentation on this issue. Last modified: 2011-08-03 WordPress ID: 1000