User-agent: MJ12bot
Disallow: /

User-agent: *
Crawl-delay: 60
Request-rate: 1/60
Visit-time: 0130-0330
Disallow: /news_posts/post/157$
Disallow: /news_posts/index/pop$
Disallow: /news_posts/type/17$
Disallow: /news_posts$
Disallow: /news_posts/index$