Tools / Lemmy Scraper: Communities, Posts, Comments & Search
Lemmy Scraper: Communities, Posts, Comments & Search
Collect posts and comments from Lemmy (the federated Reddit alternative) through each instance's public API. Input: communities, searchTerms. Output (JSON/CSV): one row per item with type, community, title, url, author, score, commentCount, published, link, isNew.
What it's for
- Brand and topic monitoring on the fediverse
- Community research for open-source / tech products
- Sentiment datasets from public discussions
Example input
{
"communities": [
"[email protected]",
"https://lemmy.ml/c/linux",
"[email protected]"
],
"searchTerms": [
"apify"
],
"maxPostsPerSource": 3,
"includeComments": true,
"maxCommentsPerPost": 2,
"monitorMode": true
}Example output
{
"type": "community",
"input": "[email protected]",
"instance": "lemmy.world",
"name": "technology",
"title": "Technology",
"actorId": "https://lemmy.world/c/technology",
"description": "This is a [most excellent](https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExN3N0NmhuODNib3d3Nzg0OHU2bTFqMXAzNW42Y2JsOTVmenNsNG8ycSZlcD12MV9p…",
"subscribers": 88730,
"posts": 22197,
"comments": 969526,
"usersActiveMonth": 14627,
"nsfw": false,
"published": "2023-06-11T02:16:17.173Z"
}Input reference
| Field | Type | Notes |
|---|---|---|
communities | array | [email protected], [email protected], or a community URL. |
searchTerms | array | Keyword search for posts (federated) on the search instance. |
searchInstance | string | Search instance |
sort | string | Sort |
maxPostsPerSource | integer | Max posts per community/search |
includeComments | boolean | Include comments |
maxCommentsPerPost | integer | Top comments. |
sinceDate | string | YYYY-MM-DD. |
monitorMode | boolean | Remember seen post ids; flag/output only new posts. |
stateStoreName | string | State store name |
Pricing
Pay per event: Post $0.0005, Comment $0.0002. Plus Apify's small per-run start fee. Apify's free plan includes monthly credit to try it.
How to run
- Open the Store page and click Try for free.
- Fill in the input (or paste the example JSON in the JSON tab).
- Click Start, then export results as CSV, Excel or JSON, or call it via the Apify API.
- Optional: add a Schedule and a webhook/Slack/email integration for ongoing monitoring.