Reddit historical data for research: what still works after Pushshift
The state of Reddit data sources, September 2026
Anyone doing research on Reddit runs into the same wall within the first hour: the official endpoints block or rate limit bulk access, and the tool everyone built on for fifteen years is gone. Here is the field as it stands now.
| Source | Status for research | What you get |
|---|---|---|
www.reddit.com/<sub>.json | Works from a residential connection, blocked or rate limited at scale | Current listings only, about 1,000 items deep, no historical window |
| Official API (OAuth) | Live, free tier at 100 requests per minute | Current posts and comments, real-time. No deep history |
| Pushshift | Closed to the public since 2023 | Was the standard: full-history dumps and search. Search UI and ingestion are gone |
| Arctic Shift API | Live and tested for this guide; no key needed | Full-history search over posts and comments, JSON, GET only |
| PullPush | Community Pushshift rewrite, uptime varies | Similar search endpoints, worth checking when Arctic Shift is throttling you |
What happened in 2023, in four dates
The reason Reddit historical data got hard is one policy shift, and the dates explain why so many guides online are now dead ends.
Spring 2023. Reddit restricted Pushshift, the volunteer archive that had mirrored every post and comment since 2015 and powered most academic research and mod tooling. Moderator access went first, then the public routes closed behind it.
April 18, 2023. Reddit announced paid API pricing: $0.24 per 1,000 calls, with a free tier capped at 100 requests per minute. The price was aimed at third-party apps, and no researcher could afford it as a bulk-collection budget either.
June 30, 2023. Apollo, Reddit is Fun, and most third-party clients shut down under the new pricing. That was the day casual bulk access to Reddit data effectively ended.
August 2023. A community project began stitching the last available Pushshift dumps back together and posting monthly archives, which is the lineage the archive below grew out of. Everything since has been volunteer-run, which is why checking a source is still alive matters before you build on it.
What the official .json endpoints actually do
The trick everyone knows is appending .json to any Reddit URL. It still works from a laptop on a home connection, and for a one-off look at a subreddit it is the fastest path. For research it fails in three predictable ways, all of which we hit from our box this month.
Datacenter IPs get blocked. From our research box on September 7, 2026, old.reddit.com/r/SideProject/new.json returned 403 before serving content, and www.reddit.com/r/SideProject/new.json returned an HTML block page: no data in the body at all, just a spinner animation and a script challenge. Reddit fronts these endpoints with bot protection, and a server IP with no cookie history trips it on the first request.
Rate limits tighten under load. From residential connections, pacing matters more than politeness. In our own runs, requests spaced 570-580 seconds apart still triggered RATELIMIT cooldowns of 16-21 seconds on the third through fifth calls in a burst. The ceiling drops exactly when your collection gets ambitious.
There is no past. Listings paginate roughly 1,000 items back per sort order. For a subreddit with any traffic, last month is already out of reach. Anything older than that needs an archive, and for research windows measured in years, the official endpoints were never the answer.
Arctic Shift: the research fallback that is still up
Arctic Shift is a volunteer archive project (github.com/ArthurHeitmann/arctic_shift) that keeps collecting Reddit posts and comments into monthly dumps and serves them through a free API at arctic-shift.photon-reddit.com. No account, no key, GET requests only. It fills the hole Pushshift left, with two honest caveats: it rate limits hard, and it depends on one project staying funded and hosted.
Three real queries, run September 7, 2026, trimmed for display. First, search posts by subreddit and title keyword:
GET https://arctic-shift.photon-reddit.com/api/posts/search?subreddit=datasets&limit=2&title=datasetHTTP 200
{"data":[{
"author":"AdkoSokdA",
"created_utc":1788703259,
"id":"1w8wzyu",
"link_flair_text":"resource",
"num_comments":0,
"permalink":"/r/datasets/comments/1w8wzyu/the_biggest_open_football_dataset_430_000/",
"score":1,
"selftext":"Hello!\n\nThis dataset I have created last year has been downloaded over **20 000 tim...",
"title":"The biggest open football dataset (430,000...",
"subreddit":"datasets"
}]}The objects are full Reddit post objects: body text, flair, timestamps, comment counts. The same post fetched by ID shows one archive-specific field, _meta.retrieved_2nd_on, the date the archive re-crawled the record:
GET https://arctic-shift.photon-reddit.com/api/posts/ids?ids=1tfpcpdHTTP 200
{"data":[{
"author":"Careful_Sand_7838",
"created_utc":1779023958,
"_meta":{"retrieved_2nd_on":1779153591},
"subreddit":"datasets",
"title":"I made the largest public gender-labeled Japanese..."
}]}And to prove the archive covers the Pushshift era itself, search the subreddit where its users went. These are posts from August 2026, still asking where Pushshift went:
GET https://arctic-shift.photon-reddit.com/api/posts/search?subreddit=pushshift&limit=3HTTP 200
[{"id":"1vvf24z","created_utc":1787411752,
"title":"Is PushShift unable to search user history if we disable comment/post visibility?..."},
{"id":"1vk5xjj","created_utc":1786321613,
"title":"What's the active samac.io equivalent for modern pushshift?"},
{"id":"1vhniup","created_utc":1786067897,
"title":"pushshift not processing forms"}]The rate limit is the one operational fact to plan around. Hammer the API and it answers with HTTP 422 and the body Timeout, slow down. Our harvest runs in stages about 2 hours apart to stay inside the ceiling. For a bounded study, that pacing is fine; for a live dashboard, this is the wrong source.
What a real harvest looks like, numbers included
To size the effort before you start: one of our own research pulls, September 2026, used the method above across 14 subreddits (SideProject, Entrepreneur, EntrepreneurRideAlong, smallbusiness, SaaS, microsaas, webdev, datasets, DataHoarder, LocalLLaMA, SillyTavernAI, indiehackers, digitalproducts, Passive_Income), with a window from March 2025 to September 2026 and title keywords like made $, MRR, sold, first sale, revenue, dataset, and template.
The funnel: 5,523 unique posts pulled from the archive, 1,243 after keyword and quality filtering, 133 complete stories worth analysis. Most of the loss is deleted content, which stays deleted in any archive: a post removed by its author shows up as [removed] with the body gone. Upvote and comment counts are snapshots from harvest time, so cite them as of the retrieval date.
Total wall-clock time was spread over staged runs, not a single sitting. That is the honest cost of a free volunteer archive: the data is deep, the pipe is narrow.
How to frame it in your own work
Treat these archives as what they are: public research collections maintained by volunteers, cited by name and retrieval date like any dataset. In a paper or report, name the archive, the endpoint or dump, and the date you pulled the rows. Keep the raw JSON as your snapshot, because live scores and edited bodies will drift away from what you analyzed. And if your export becomes a dataset others will touch, document its columns the way you would for any CSV; a data dictionary is a one-hour job that saves every downstream user the guessing. Deciding between keeping the export as a file versus wiring a query layer over it is the same tradeoff as any other source: the CSV vs API decision table covers it.
FAQ
Is Pushshift still available?
No public access, and it has been that way since 2023. Reddit disabled Pushshift moderator access in spring 2023 and closed the remaining routes that year. The search UI and ingestion are gone. The Arctic Shift API covers the same need for research: full-history search over posts and comments, free, no key.
Can I get Reddit data through the official API?
Yes, for current data. The free OAuth tier allows 100 requests per minute, and the plain .json endpoints work from a normal browser connection. Two limits matter for research: there is no deep history, listings paginate only about 1,000 items back, and the plain .json endpoints often return 403 or a block page when called from datacenter IPs. Bulk historical collection needs an archive instead.
What is the Arctic Shift API?
A public Reddit archive and free API at arctic-shift.photon-reddit.com, built on the same idea as Pushshift. It serves full post and comment objects as JSON over GET endpoints with no key: search by subreddit, date, title, and more, or fetch specific IDs. It rate limits aggressively and returns 422 Timeout, slow down errors when you hammer it, so harvests run best in spaced stages.