History

kev 25a5e67907 update crawlee		2024-04-10 17:30:24 +08:00
..
.dockerignore	add crawlee	2023-03-08 15:03:19 +08:00
README.md	update crawlee	2024-04-10 17:30:24 +08:00
docker-compose.yml	update crawlee	2023-03-08 15:51:09 +08:00

README.md

crawlee

Crawlee is a web scraping and browser automation library Crawlee is a web scraping and browser automation library.

$ docker run --rm -it -v $PWD:/tmp apify/actor-node:20 sh
>>> export PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1
>>> export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1
>>> npx crawlee create -t cheerio-js my-crawler
>>> mv my-crawler /tmp
>>> exit

$ docker-compose build my-crawler

$ docker-compose run --rm my-crawler

$ tree my-crawler/storage/
├── datasets
│   └── default
│       └── 000000001.json
├── key_value_stores
└── request_queues