Bug fixes

• Fix a cryptic traceback when a pipeline fails on open_spider() (#2011)
• Fix embedded IPython shell variables (fixing #396 that re-appeared in 1.2.0, fixed in #2418)
• A couple of patches when dealing with robots.txt:
  • handle (non-standard) relative sitemap URLs (#2390)
  • handle non-ASCII URLs and User-Agents in Python 2 (#2373)

Documentation

• Document "download_latency" key in Request‘s meta dict (#2033)
• Remove page on (deprecated & unsupported) Ubuntu packages from ToC (#2335)
• A few fixed typos (#2346, #2369, #2369, #2380) and clarifications (#2354, #2325, #2414)

Other changes

• Advertize conda-forge as Scrapy’s official conda channel (#2387)
• More helpful error messages when trying to use .css() or .xpath() on non-Text Responses (#2264)
• startproject command now generates a sample middlewares.py file (#2335)
• Add more dependencies’ version info in scrapy version verbose output (#2404)
• Remove all *.pyc files from source distribution (#2386)