WIP pre-release for the upcoming ArchiveBox v0.8.0 release.

[!WARNING] This is an unfinished alpha pre-release. We're promoting it a little earlier than usual because it contains a ✨ big Django upgrade ✨ that affects many areas of the codebase, and we want brave early adopters to help us test it! If that sounds like you, make sure to back up your archive first, then give it a try and let us you if you find any bugs by opening a new issue!

Try this release early using docker or pip:

# with docker (pre-built)
docker pull archivebox/archivebox:dev
# with docker (built from source)
docker build -t archivebox:dev https://github.com/ArchiveBox/ArchiveBox.git#dev
# with pip (built from source)
pip install 'git+https://github.com/pirate/ArchiveBox@dev'

To use the new noVNC container to view & control the ArchiveBox browser remotely, grab the updated docker-compose.yml and follow these instructions.

Highlights

• upgrade to Django 4.2 (thanks @jimwins!)
• add new _EXTRA_ARGS options (thanks @benmuth!)
• add new generic_jsonl parser (thanks @jimwins!)
• switch to feedparser for RSS parsing (thanks @jimwins!)
• remember Snapshot detail page header expanded/collapsed state
• allow more restrictive NFS permission coercion on ./data/archive
• check /, /data, and /data/archive in Docker and warn if running low on disk space
• fix /browsers chown on Docker armv7 entrypoint failing
• disable chrome automatic self-updating when running headless
• Add ability to populate is_staff and is_superuser flags during LDAP first auth
• add gitea and other domains to default GIT_DOMAINS list to run git archiving on
• bump yt-dlp and singlefile versions
• fix RESOLUTION being ignored when using Chrome headless in Docker
• fix sorting by Size / Files in the Admin Snapshots list page UI
• fix spinner icon showing on some Snapshots instead of favicon when only a few extractors are enabled
• fix yt-dlp sometimes failing to archive media due to filenames being too long or containing special characters
• fix wget extractor not finding output when :80 or :443 port is present in the original URL
• fix /var/spool/cron/crontabs permissions when mounting it via Docker

• COMING SOON: new sci-dl scientific paper downloader being worked on by @benmuth

What's Changed

• Fallback to not chown'ing ./data/archive dir if it's a network mount that prevents ownership changes by @gnattu in https://github.com/ArchiveBox/ArchiveBox/pull/1312
• Make "DOCS" link in navigation bar open in new tab by @speerer in https://github.com/ArchiveBox/ArchiveBox/pull/1323
• Fixes #1193 Bug: Search sometimes shows the same snapshot twice by @neel-suthar in https://github.com/ArchiveBox/ArchiveBox/pull/1330
• Fix for Bug: Search sometimes shows the same snapshot twice #1193 by @neel-suthar in https://github.com/ArchiveBox/ArchiveBox/pull/1331
• Show the upgrade notification only in specific views by @benmuth in https://github.com/ArchiveBox/ArchiveBox/pull/1314
• ability to populate is_staff and is_superuser flags at LDAP authentication by @vladimirdulov in https://github.com/ArchiveBox/ArchiveBox/pull/1335
• Update README.md by @pirate in https://github.com/ArchiveBox/ArchiveBox/pull/1336
• Fix generic_json parser by @jimwins in https://github.com/ArchiveBox/ArchiveBox/pull/1365
• Make it a little easier to run specific tests by @jimwins in https://github.com/ArchiveBox/ArchiveBox/pull/1371
• Add COOKIES_FILE support for singlefile extractor by @naoph in https://github.com/ArchiveBox/ArchiveBox/pull/1372
• Update README.md Browser Extension link by @rdela in https://github.com/ArchiveBox/ArchiveBox/pull/1374
• Use feedparser for RSS parsing by @jimwins in https://github.com/ArchiveBox/ArchiveBox/pull/1362
• Use COOKIES_FILE to fetch page titles by @benmuth in https://github.com/ArchiveBox/ArchiveBox/pull/1364
• Add _EXTRA_ARGS for various extractors by @benmuth in https://github.com/ArchiveBox/ArchiveBox/pull/1360
• Add generic_jsonl parser by @jimwins in https://github.com/ArchiveBox/ArchiveBox/pull/1370
• Revise md section not formatting properly in html by @n-hebert in https://github.com/ArchiveBox/ArchiveBox/pull/1382
• Update to Django 4.2.x by @jimwins in https://github.com/ArchiveBox/ArchiveBox/pull/1388

New Contributors

• @gnattu made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1312
• @speerer made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1323
• @neel-suthar made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1330
• @jimwins made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1365
• @naoph made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1372
• @rdela made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1374
• @n-hebert made their first contribution in https://github.com/ArchiveBox/ArchiveBox/pull/1382

Full Changelog: https://github.com/ArchiveBox/ArchiveBox/compare/v0.7.2...v0.8.0-rc