Genieo/1.0 のDoSアタック
ブラウザプラグインらしい。
- robots.txtを無視してアクセスする。
Connection: closeで間隔をおかずに連続アクセスする。- URLの
#以降を無視しないので同じURLを何度も読み込む。
robots.txtの件は Disallow: /nobot と書いても /nobot.html にアクセスする。仕様にあるような先頭一致ではなく / 区切りで完全一致させているのだろうか。
ユーザ環境でクロールを行ってリコメンデーションをするらしいが、これだけトラフィックが発生するとユーザ側も重くなるのではと思う。使ってないので実際は不明なもののマルウェアとして分類しているところもある。
2012年頃はUser-Agentの偽装もやっていたらしいのでそれに比べれば今はUser-Agentでブロックできるだけマシなのだろう。
以下ログ
* - - [...8:59 +0900] "GET /robots.txt HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...8:59 +0900] "GET /page HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...8:59 +0900] "GET /pagea HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...8:59 +0900] "GET /pageb HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...8:59 +0900] "GET /pagea HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:00 +0900] "GET /pagea-49 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:00 +0900] "GET /pagea-7 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:00 +0900] "GET /pagea-48 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:00 +0900] "GET /pagea-6 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:01 +0900] "GET /pagea-47 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:01 +0900] "GET /pagea-9 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:01 +0900] "GET /pagea-46 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:01 +0900] "GET /pagea-8 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:01 +0900] "GET /nobot-page HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:02 +0900] "GET /pagea-1 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:03 +0900] "GET /pagea-3 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:03 +0900] "GET /pagea-2 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:04 +0900] "GET /pagea-5 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:04 +0900] "GET /pagea-4 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:04 +0900] "GET /pagea-40 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:05 +0900] "GET /pagea-41 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:05 +0900] "GET /pagea-44 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:06 +0900] "GET /pagec HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:07 +0900] "GET /pagea-45 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:08 +0900] "GET /pagea-42 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:08 +0900] "GET /pagea-43 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:09 +0900] "GET /pagea-36 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:09 +0900] "GET /pagea-35 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:10 +0900] "GET /pagea-38 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:10 +0900] "GET /pagea-37 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:11 +0900] "GET /pagea/sub HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:11 +0900] "GET /pagea-39 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:12 +0900] "GET /pagea?query HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:12 +0900] "GET /pagea-30 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; ®1.0 http://www.genieo.com/webfilter.html)" * - - [...9:12 +0900] "GET /pagea-31 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:13 +0900] "GET /pagea-32 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:13 +0900] "GET /pagea-33 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:14 +0900] "GET /pagea-34 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:15 +0900] "GET /pagea/sub2 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:15 +0900] "GET /pagea-28 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:16 +0900] "GET /pagea-29 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:17 +0900] "GET /pagea-26 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:18 +0900] "GET /pagea-27 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:19 +0900] "GET /pagea-24 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:19 +0900] "GET /paged HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:20 +0900] "GET /pagea-25 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:20 +0900] "GET /pagea-23 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:20 +0900] "GET /pagea-22 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:21 +0900] "GET /pagea-21 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:21 +0900] "GET /pagea-20 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:22 +0900] "GET /pagea?query2 HTTP/1.1" 200 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:23 +0900] "GET /pagee HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:23 +0900] "GET /pagea-17 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:24 +0900] "GET /pagea-18 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:24 +0900] "GET /pagea-19 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:24 +0900] "GET /pagea-13 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:25 +0900] "GET /pagea-14 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:25 +0900] "GET /pagea-15 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:26 +0900] "GET /pagea-16 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:26 +0900] "GET /pagea-10 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:27 +0900] "GET /pagea-12 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:27 +0900] "GET /pagea-11 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:28 +0900] "GET /pagea/sub HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:29 +0900] "GET /pagea-50 HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)" * - - [...9:30 +0900] "GET /pagea/sub#frag HTTP/1.1" 429 * "-" "Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)"
bot禁止領域にアクセスしたことで制限が発生して一定時間429レスポンスが返るようになっている。
