Commit graph

4590 commits

Author SHA1 Message Date
Nick Sweeting
098573705b
clean up Dockerfile 2024-12-18 02:24:21 -08:00
Nick Sweeting
1e7b1dff83
move Dockerfile.simple to ArchiveBox/docker-archivebox/README.md 2024-12-18 01:35:43 -05:00
Nick Sweeting
0ad1bdaf93
remove old deprecated bin/archive entrypoint 2024-12-18 01:31:38 -05:00
Nick Sweeting
90f511c367
Bump Dockerfile.simple to rc51 2024-12-18 01:30:37 -05:00
Nick Sweeting
c54b944674
change docker build to use uv exclusively instead of requirements.txt 2024-12-17 21:09:41 -08:00
Nick Sweeting
b4c5004c86
Update README.md
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Build Docker image / buildx (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2024-12-17 22:53:05 -05:00
Nick Sweeting
2b77422d5b
remove requirements.txt entirely because people keep trying to run it for some reason
guys this file is regenrated by our CI process and should not be used for anything other than the CI process
2024-12-17 22:43:05 -05:00
Nick Sweeting
f31adff0ed
Update README.md
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Build Debian package / build (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
2024-12-15 03:36:12 -05:00
Nick Sweeting
f6d22a3cc4
tweak worker updated logic and add output_dir_template and symlinks logic 2024-12-13 06:03:52 -08:00
Nick Sweeting
34e4b48557
add example js extractor 2024-12-12 22:15:17 -08:00
Nick Sweeting
74e08a18aa
add filestore migrations 2024-12-12 22:15:17 -08:00
Nick Sweeting
c11a1b54f1
add new worker test 2024-12-12 22:08:18 -08:00
Nick Sweeting
5c06b8ff00
add new Event model to workers/models 2024-12-12 22:08:17 -08:00
Nick Sweeting
651ba0b11c
add new Process model to Machine models 2024-12-12 21:45:55 -08:00
Nick Sweeting
2a1afcf6c2
move crawl models back into dedicated app 2024-12-12 21:45:55 -08:00
Nick Sweeting
bd5dd2f949
clearer core models separation of concerns using new basemodels 2024-12-12 21:45:53 -08:00
Nick Sweeting
930b9bf386
add archivebox worker cli cmd to list of all cmds 2024-12-12 21:44:44 -08:00
Nick Sweeting
bab26d6a9b
better base_models separation of concerns 2024-12-12 21:44:43 -08:00
Nick Sweeting
51447b9d0a
bump django version to 5.1.4 2024-12-12 21:42:15 -08:00
Nick Sweeting
6b3e297db8
fix lock_pkgs.sh version parsing and python version 2024-12-12 21:41:45 -08:00
Nick Sweeting
5cf7725f0e
add new archivebox worker implementation based on better distributed systems principles 2024-12-12 21:41:45 -08:00
Nick Sweeting
a859278a63
tags apps.py 2024-12-12 21:41:45 -08:00
Nick Sweeting
1444cf7fda
add new KVTags system 2024-12-12 21:41:44 -08:00
Nick Sweeting
81bf81ab10
add extract.js prototype extractor
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Build Debian package / build (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
2024-12-06 02:06:40 -08:00
Nick Sweeting
ac53fdf677
make chrome binary and configs directly runnable and make extractor use external bin 2024-12-06 02:06:39 -08:00
Nick Sweeting
a572db307b
fix syntax errors (#1609)
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Build Docker image / buildx (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2024-12-05 19:36:37 -05:00
dish
f1b9aec873
fix syntax errors 2024-12-05 13:52:33 -05:00
Nick Sweeting
d192eb5c48
add filestore content addressible store draft
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Build Debian package / build (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
2024-12-04 02:15:04 -08:00
Nick Sweeting
dc0f1b0efc
add new File model in filestore 2024-12-04 02:15:04 -08:00
Nick Sweeting
a3fe78afaa
add basename to hashing get_dir_info 2024-12-04 02:15:04 -08:00
Nick Sweeting
73a75bb4c9
Update FUNDING.yml 2024-12-04 01:38:07 -08:00
Nick Sweeting
8c8ec6aff0
add extractors README
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Build Docker image / buildx (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2024-12-03 02:15:17 -08:00
Nick Sweeting
dcd7e2555e
add new archivebox_extract cli command 2024-12-03 02:14:56 -08:00
Nick Sweeting
337acdac9c
add base extractor class 2024-12-03 02:14:42 -08:00
Nick Sweeting
1ceaa1ac7a
add ABID model check and fix model inheritance 2024-12-03 02:14:21 -08:00
Nick Sweeting
c374d7695e
allow getting crawl from API as rss feed 2024-12-03 02:13:45 -08:00
Nick Sweeting
eae7ed8447
add hashing misc library for merkle tree generation 2024-12-03 02:12:20 -08:00
Nick Sweeting
22901406aa
Update 2-feature_request.yml
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Build Debian package / build (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
2024-11-22 18:29:58 -05:00
Nick Sweeting
44d337a167
convert index.schema.ArchiveResult and Link to pydantic
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Build Debian package / build (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
2024-11-19 06:32:48 -08:00
Nick Sweeting
b948e49013
add urls log to Crawl model 2024-11-19 06:32:33 -08:00
Nick Sweeting
28386ff172
add jobs_dashboard.html back 2024-11-19 05:35:52 -08:00
Nick Sweeting
4dd53dc12a
Merge branch 'newchanges' into dev
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Build Docker image / buildx (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2024-11-19 05:28:20 -08:00
Nick Sweeting
b852951c58
fix cli loading edge case where setup_django wasnt running when it should 2024-11-19 05:27:35 -08:00
Nick Sweeting
6b47510f70
always pre-setup binproviders 2024-11-19 05:24:12 -08:00
Nick Sweeting
f8e2f7c753
restore missing archivebox_update work 2024-11-19 05:09:19 -08:00
Nick Sweeting
52446b86ba
restore missing archivebox_status work 2024-11-19 05:08:41 -08:00
Nick Sweeting
0f536ff18b
restore missing archivebox_schedule work 2024-11-19 05:07:55 -08:00
Nick Sweeting
fe3320eff0
restore missing archivebox_remove work 2024-11-19 05:07:12 -08:00
Nick Sweeting
230bf34e14
restore missing archivebox_config work 2024-11-19 05:05:06 -08:00
Nick Sweeting
ee548eb16e
fix archivebox install not using LIB_DIR 2024-11-19 04:44:43 -08:00