GoogleBot

A recent study by Incapsula reports that 61.5% of all website traffic is now generated by bots. The findings from the security firm show this as a 21% rise over last year.  While some of the automated programs are malicious, Incapsula reports that the largest growth in traffic actually comes from “good” bots which are used by search engines to help index content.

The website traffic report was generated as part of a 90 day study that observed over 1.45 billion bot visitors to 20,000 different websites.  30 percent of web traffic comes from malicious bots.  These bots are broken into categories such as scrapers, which copy and steal a website’s content, hackers that steal credit cards or inject malware, spammers posting fraudulent links or fake comments and impersonator bots that gather intelligence or launch DDoS attacks.  While 30% seems like a lot of activity, Incapsula stressed the fact that this is actually a decrease in activity over past years.  Link spam, in particular, has seen a huge decrease in activity and has dropped 75% over last year.  The security firm speculates that this drop is in large part due to increased security measures implemented by Google.

BotTraffic

Graphic by Incapsula

The traffic study also shines some light on a potentially distressing trend.  While the overall number of “bad” bots has decreased, the number of sophisticated attackers has gone up.  8% of the bots are classified as “Other Impersonators”, a group of unclassified bots that try to assume someone else’s identity.  Incapsula gave the example, “some of these bots try to pass themselves as search engine bots or agents of other legitimate services. The goal is always the same – to infiltrate their way through the website’s security measures… Where other

As internet bots become more sophisticated and users continue to invent new ways to receive and share information the growth of autonomous bots will likely grow even further.  Already, many users rely on bots to receive news headlines delivered to their inboxes or home screens and we would all be lost without Google.  In the future, human internet users better get used to being a minority in a very bot dominant world.malicious bots are agents of known malware …these “Impersonators” are custom-made bots, usually crafted for a very specific malicious activity.”  The reason this gets special mention is that it means that malware is no longer created only as a wide net to catch unprepared or ignorant users. Instead, malware is now created specifically to break through security systems and bypass defenses.

– Richard Keene
IT Computer Support of New York
Webmaster and Lead Designer