Commit graph

497 commits

Author SHA1 Message Date
Nick Sweeting
7975b47c85
remove dependencies on unneeded libraries 2024-12-18 18:07:35 -08:00
dish
9ca66c6a2b
fix syntax error in archivebox/core/models.py 2024-12-18 18:17:17 -05:00
Nick Sweeting
f6d22a3cc4
tweak worker updated logic and add output_dir_template and symlinks logic 2024-12-13 06:03:52 -08:00
Nick Sweeting
5c06b8ff00
add new Event model to workers/models 2024-12-12 22:08:17 -08:00
Nick Sweeting
2a1afcf6c2
move crawl models back into dedicated app 2024-12-12 21:45:55 -08:00
Nick Sweeting
bd5dd2f949
clearer core models separation of concerns using new basemodels 2024-12-12 21:45:53 -08:00
Nick Sweeting
ac53fdf677
make chrome binary and configs directly runnable and make extractor use external bin 2024-12-06 02:06:39 -08:00
Nick Sweeting
d192eb5c48
add filestore content addressible store draft
Some checks failed
CodeQL / Analyze (python) (push) Has been cancelled
Build Debian package / build (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
2024-12-04 02:15:04 -08:00
Nick Sweeting
1ceaa1ac7a
add ABID model check and fix model inheritance 2024-12-03 02:14:21 -08:00
Nick Sweeting
c374d7695e
allow getting crawl from API as rss feed 2024-12-03 02:13:45 -08:00
Nick Sweeting
b948e49013
add urls log to Crawl model 2024-11-19 06:32:33 -08:00
Nick Sweeting
2595139180
improve statemachine logging and archivebox update CLI cmd 2024-11-19 03:31:05 -08:00
Nick Sweeting
c9a05c9d94
working archivebox update CLI cmd 2024-11-19 02:32:05 -08:00
Nick Sweeting
5f01fc8307
fix archivebox shell and manage CLI commands 2024-11-19 00:48:39 -08:00
Nick Sweeting
328eb98a38
move main funcs into cli files and switch to using click for CLI 2024-11-19 00:18:51 -08:00
Nick Sweeting
569081a9eb
rename abid_utils to base_models
Some checks are pending
CodeQL / Analyze (python) (push) Waiting to run
Build Debian package / build (push) Waiting to run
Build Docker image / buildx (push) Waiting to run
Deploy static content to Pages / deploy (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Build GitHub Pages website / build (push) Waiting to run
Build GitHub Pages website / deploy (push) Blocked by required conditions
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2024-11-18 19:40:05 -08:00
Nick Sweeting
65afd405b1
merge seeds and crawls apps 2024-11-18 19:23:14 -08:00
Nick Sweeting
4c25e90378
move monkey_patches.py into archivebox.misc subfolder 2024-11-18 19:10:42 -08:00
Nick Sweeting
4a5d607296
move logging_util into archivebox.misc subfolder 2024-11-18 19:08:49 -08:00
Nick Sweeting
e469c5a344
merge queues and actors apps into new workers app 2024-11-18 18:52:48 -08:00
Nick Sweeting
0acd388c02
fix imports and deps 2024-11-18 18:07:34 -08:00
Nick Sweeting
eeb2671e4d
API improvements 2024-11-18 04:27:38 -08:00
Nick Sweeting
eb53145e4e
working state machine flow yay 2024-11-18 04:27:38 -08:00
Nick Sweeting
9adfe0e2e6
add code to log all SQL queries for DEBUG 2024-11-18 04:27:38 -08:00
Nick Sweeting
385ccaa14d
extend core models with ModelWithOutputDir 2024-11-18 04:27:38 -08:00
Nick Sweeting
f5727c7da2
rename actors to workers 2024-11-18 04:27:37 -08:00
Nick Sweeting
1ec2753664
fix statemachine create_root_snapshot and retry timing 2024-11-18 04:27:37 -08:00
Nick Sweeting
36d24cd8d7
add jobs dashboard 2024-11-17 20:09:55 -08:00
Nick Sweeting
8f8fbbb7a2
API fixes and add actors endpoints 2024-11-17 20:09:06 -08:00
Nick Sweeting
c8e186f21b
fix plugin loading order, admin, abx-pkg
Some checks failed
Run linters / lint (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Build Debian package / build (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
2024-11-16 06:44:12 -08:00
Nick Sweeting
ba26d75079
add notes and label fields, fix model getters 2024-11-16 02:47:35 -08:00
Nick Sweeting
227fd4e1c6
fix statemachine progression for Snapshot, Crawl, and ArchiveResult 2024-11-16 02:46:45 -08:00
Nick Sweeting
ed43f1d027
better docstrings and comments 2024-11-15 23:21:40 -08:00
Nick Sweeting
a9a3b153b1
more StateMachine, Actor, and Orchestrator improvements
Some checks failed
Build Debian package / build (push) Has been cancelled
Build GitHub Pages website / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Deploy static content to Pages / deploy (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build GitHub Pages website / deploy (push) Has been cancelled
2024-11-04 07:08:39 -08:00
Nick Sweeting
48f8416762
add new core and crawsl statemachine manager
Some checks failed
Build Docker image / buildx (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
Build Debian package / build (push) Has been cancelled
2024-11-03 00:41:11 -07:00
Nick Sweeting
9b24fe7390
merge dev 2024-11-02 17:34:33 -07:00
Nick Sweeting
721427a484
hide progress bar on startup
Some checks failed
Build Debian package / build (push) Has been cancelled
Build Docker image / buildx (push) Has been cancelled
Build Homebrew package / build (push) Has been cancelled
Run linters / lint (push) Has been cancelled
Build Pip package / build (push) Has been cancelled
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Has been cancelled
Run tests / docker_tests (push) Has been cancelled
2024-10-31 07:11:15 -07:00
Nick Sweeting
70926f1d9f
replace os.access with os.path.isdir 2024-10-28 19:06:38 -07:00
Nick Sweeting
d93aa46949
fix django.forms.JSONField does not exist 500 error 2024-10-28 18:47:45 -07:00
Nick Sweeting
b3c1cb716e
move abx plugins inside vendor dir
Some checks are pending
Build Debian package / build (push) Waiting to run
Build Docker image / buildx (push) Waiting to run
Build Homebrew package / build (push) Waiting to run
Run linters / lint (push) Waiting to run
Build Pip package / build (push) Waiting to run
Run tests / python_tests (ubuntu-22.04, 3.11) (push) Waiting to run
Run tests / docker_tests (push) Waiting to run
2024-10-28 04:07:35 -07:00
Nick Sweeting
5d9a32c364
wip 2024-10-25 01:06:12 -07:00
Nick Sweeting
b61f6ff8d8
rename system_tasks queue to commands queue 2024-10-24 15:41:43 -07:00
Nick Sweeting
9e40dd69a4
more config improvements, move away from settings GLOBALS to getters 2024-10-24 14:50:07 -07:00
Nick Sweeting
312e40b95b
finally get rid of config/legacy in favor of configfile.py and django.py 2024-10-21 03:06:19 -07:00
Nick Sweeting
7a6f1f36d2
trigger abx.pm.hook.ready from core.AppConfig.ready 2024-10-21 01:31:02 -07:00
Nick Sweeting
d48a99ab64
only load django huey config when not getting help or version to avoid creating queue.sqlite3 2024-10-21 00:38:17 -07:00
Nick Sweeting
02a1fc3049
rename sessions app in INSTALLED_APPS to personas 2024-10-21 00:37:57 -07:00
Nick Sweeting
c9f88f5875
add sessions and seeds models 2024-10-15 22:32:13 -07:00
Nick Sweeting
80d8a6b667
split archivebox.use into archivebox.reads and archivebox.writes 2024-10-15 01:03:01 -07:00
Nick Sweeting
aaf069fab0
remove tags field from Machine admin 2024-10-15 01:02:13 -07:00