1425 commits

Author	SHA1	Message	Date
Nick Sweeting	11b067a1ae	Merge branch 'dev' into plugins-browsertrix	2024-03-15 15:45:56 -07:00
jim winstead	5478d13d52	Add generic_jsonl parser ... Resolves #1369	2024-03-14 15:42:29 -07:00
Nick Sweeting	ca2c484a8e	Add _EXTRA_ARGS for various extractors (#1360)	2024-03-14 01:55:09 -07:00
Nick Sweeting	48f4b12ae2	Use COOKIES_FILE to fetch page titles (#1364)	2024-03-14 01:52:44 -07:00
Nick Sweeting	099f7d00fe	Use feedparser for RSS parsing (#1362) ... Fixes #1171 Fixes #870 (probably, would need to test against a Wallabag Atom file to Fixes #135 Fixes #123 Fixes #106	2024-03-14 01:51:45 -07:00
Nick Sweeting	3512dc7e60	Disable searching for existing chrome user profiles by default	2024-03-14 00:58:45 -07:00
Ben Muthalaly	f4deb97f59	Add ARGS and EXTRA_ARGS for Mercury extractor	2024-03-05 21:15:38 -06:00
Ben Muthalaly	d8cf09c21e	Remove unnecessary variable length args for dedupe	2024-03-05 21:13:45 -06:00
Ben Muthalaly	5082d61613	Merge branch 'title-cookies-file' of https://github.com/benmuth/ArchiveBox into title-cookies-file	2024-03-05 02:03:03 -06:00
Ben Muthalaly	4686da91e6	Fix cookies being set incorrectly	2024-03-05 01:48:35 -06:00
Ben Muthalaly	d74ddd42ae	Flip dedupe precedence order	2024-03-01 14:50:32 -06:00
jim winstead	9f462a87a8	Use feedparser for RSS parsing in generic_rss and pinboard_rss parsers ... The feedparser packages has 20 years of history and is very good at parsing RSS and Atom, so use that instead of ad-hoc regex and XML parsing. The medium_rss and shaarli_rss parsers weren't touched because they are probably unnecessary. (The special parse for pinboard is just needing because of how tags work.) Doesn't include tests because I haven't figured out how to run them in the docker development setup. Fixes #1171	2024-03-01 11:25:45 -08:00
Nick Sweeting	a577d1ed23	Merge branch 'dev' into title-cookies-file	2024-02-29 21:29:36 -08:00
Nick Sweeting	fe11e1c2f4	check if COOKIE_FILE is file	2024-02-28 18:19:44 -08:00
jim winstead	ccabda4c7d	Handle list of tags in JSON, and be more clever about comma vs. space	2024-02-28 17:38:49 -08:00
jim winstead	178e676e0f	Fix JSON parser by not always mangling the input ... Rather than by assuming the JSON file we are parsing has junk at the beginning (which maybe only used to happen?), try parsing it as-is first, and then fall back to trying again after skipping the first line Fixes #1347	2024-02-27 14:48:19 -08:00
Ben Muthalaly	68326a60ee	Add cookies file to http request in download_url	2024-02-27 15:30:31 -06:00
Ben Muthalaly	4d9c5a7b4b	Add CHROME_EXTRA_ARGS ... Also fix `YOUTUBEDL_EXTRA_ARGS`.	2024-02-23 18:40:03 -06:00
Ben Muthalaly	ab8f395e0a	Add YOUTUBEDL_EXTRA_ARGS	2024-02-23 15:40:31 -06:00
Ben Muthalaly	4e69d2c9e1	Add EXTRA_*_ARGS for wget, curl, and singlefile	2024-02-22 23:04:11 -06:00
Nick Sweeting	1ea7ac168a	Merge branch 'dev' into plugins-browsertrix	2024-02-22 04:51:31 -08:00
Nick Sweeting	8b9bc3dec8	minor fixes	2024-02-22 04:50:22 -08:00
Nick Sweeting	6a4e568d1b	new archivebox update speed improvements	2024-02-22 04:50:22 -08:00
Nick Sweeting	edabc47828	Merge branch 'dev' into plugins-browsertrix	2024-02-12 21:28:58 -08:00
Nick Sweeting	903c72fa88	fix typing errors	2024-02-12 21:28:52 -08:00
Nick Sweeting	fbb3c84be2	Merge branch 'dev' into plugins-browsertrix	2024-02-12 21:26:44 -08:00
Nick Sweeting	91c4641199	skip dir size calculation when path is too long	2024-02-12 21:26:34 -08:00
Nick Sweeting	19aefc85e6	fix get_system_user failing on uid 999 in k3s	2024-02-08 18:58:12 -08:00
Nick Sweeting	777694eb1a	add type hints to plugin config models	2024-02-08 05:55:02 -08:00
Nick Sweeting	97b185987d	add TODO to support archive.org-style urls	2024-02-08 05:54:34 -08:00
Nick Sweeting	b56bfe57f1	add CUSTOM_TEMPLATES=/data/templates default config in Docker	2024-02-08 05:54:18 -08:00
Nick Sweeting	c6faa9ab76	add extra information to headers extractor output	2024-02-06 22:22:03 -08:00
Nick Sweeting	a4bd441077	Update config.py fix trim-filenames	2024-01-31 01:59:43 -08:00
Nick Sweeting	308b493469	add pudb and type hints	2024-01-28 02:57:51 -08:00
Nick Sweeting	9861a4fb48	more mypy fixes	2024-01-28 02:57:51 -08:00
Nick Sweeting	eaa4a9c5ea	fix django core auth	2024-01-28 02:57:51 -08:00
Nick Sweeting	beb83f292d	mypy fixes	2024-01-28 02:57:51 -08:00
Nick Sweeting	4aee3b590e	Merge branch 'dev' into plugins-browsertrix	2024-01-24 12:34:41 -08:00
Nick Sweeting	e8772513ca	Update archivebox/core/auth_ldap.py ... add log line	2024-01-24 11:00:42 -08:00
Vladimir D	914df75ee7	Merge branch 'dev' into feature/is_staff-is_superuser-flags-defaults	2024-01-24 22:25:27 +04:00
Vladimir D	95580ee743	populate is_staff and is_superuser flags at LDAP authentication	2024-01-24 22:18:02 +04:00
Nick Sweeting	8e41aec099	fix plugin loading and admin config display	2024-01-24 02:15:02 -08:00
Nick Sweeting	d0e3c9502e	add defaults and system plugins	2024-01-24 00:07:07 -08:00
Nick Sweeting	0c878eb754	Merge branch 'dev' into plugins-browsertrix	2024-01-23 16:35:23 -08:00
Nick Sweeting	3b36928bdc	add yt-dlp restrict-filenames option to make filenames shorter	2024-01-23 14:20:59 -08:00
Nick Sweeting	2ba72f14ba	Merge branch 'dev' into plugins-browsertrix	2024-01-23 13:52:08 -08:00
Nick Sweeting	93781c58ce	add more git clone domains	2024-01-23 13:51:56 -08:00
Nick Sweeting	d2ec6d7ff7	Merge branch 'dev' into plugins-browsertrix	2024-01-23 10:29:41 -08:00
Nick Sweeting	c7cab7cadc	Show the upgrade notification only in specific views (#1314)	2024-01-23 10:29:14 -08:00
Nick Sweeting	7aa0dd1571	Merge branch 'dev' into plugins-browsertrix	2024-01-19 18:38:14 -08:00