Documentation/Blog/Code Recipe

How to Fetch Subreddit Posts as JSON in TypeScript

A step-by-step developer guide to extracting subreddit submissions as clean, strongly typed JSON in Node.js and TypeScript. Learn how to configure sort modes, filter by time, execute parallel queries across communities, and export datasets directly to disk.

SS
SubScraper Team
Published March 12, 2026
8 min read
Topic:fetch subreddit posts json

Why Fetching Subreddit Posts via the Direct Reddit API Fails

Every developer building an AI agent, market research pipeline, or social monitor starts by trying the simplest trick in web development: appending .json to a public Reddit URL. For instance, querying https://www.reddit.com/r/technology/hot.json works seamlessly for five or ten requests on your personal laptop.

However, the moment you move that code into a Node.js backend, a Next.js server route, AWS Lambda, or a scheduled scraper job, the direct approach breaks catastrophically. Here is why:

Severe IP Rate Limits & 429s

Reddit limits unauthenticated requests to approximately 10 to 60 queries per minute per IP. Datacenter IP blocks (AWS EC2, Google Cloud, DigitalOcean, and Vercel) are flagged and throttled with HTTP 429 Too Many Requests almost immediately.

OAuth App Hurdles & Pricing

Reddit's official Data API requires registering an official developer app, awaiting enterprise approvals, handling complicated OAuth refresh tokens, and paying prohibitive commercial rates ($0.24 per 1,000 API calls) under restrictive data terms.

CORS & Frontend Restrictions

Reddit explicitly omits Cross-Origin Resource Sharing (Access-Control-Allow-Origin) headers on public JSON routes. Client-side browser invocations fail with cross-origin browser security rejections.

Cloudflare TLS Fingerprinting

Reddit employs deep bot-defense heuristics. Headless HTTP clients without authentic browser TLS fingerprints receive silent CAPTCHA redirects or HTTP 403 Forbidden pages instead of valid JSON.

The SubScraper Solution: SubScraper provides an enterprise-ready, unblocked gateway. Every request is automatically routed through a global residential proxy pool, normalized into clean, typed TypeScript payloads, and delivered in under 800ms without managing OAuth credentials or proxy servers.
Step 1

Quick Start — Fetch Posts in 3 Lines

Getting started with the official SubScraper TypeScript SDK requires zero configuration boilerplate. First, install the package into your project:

terminal
bash
# Install the official SubScraper SDK
npm install @subscraper/sdk

Next, instantiate SubScraper with your API key and call getCommunityPosts. Here is the complete, working example:

quickstart.ts
typescript
import { SubScraperClient } from '@subscraper/sdk';

const client = new SubScraperClient({ apiKey: process.env.SUBSCRAPER_API_KEY! });
const feed = await client.getCommunityPosts({ subreddit: 'typescript', limit: 10 });

console.log(`Fetched ${feed.items?.length} posts from r/${feed.subreddit}`);

What happens in these three lines:

  1. Client Authentication: Instantiating the SDK configures persistent HTTP headers with your Bearer authentication token and sets up automatic timeout handling (default 30 seconds).
  2. Proxy Routed Query: The SDK sends an authenticated POST request to /api/v1/community-posts. SubScraper routes through a clean residential IP, resolves Reddit's anti-bot protections, and retrieves the live feed.
  3. Normalized Payload: The response is automatically parsed and mapped into strongly typed objects (feed.items), complete with titles, authors, scores, permalinks, and comment tallies.
Step 2

Available Sort Options

Reddit communities feature distinct algorithmic feeds. Passing the sort parameter lets you tailor your data ingestion for viral content, real-time alerts, or historical backfills.

Sort Optionsort paramRanking MechanismTime FilterWhen to Use Each
Hot'hot' (default)Reddit's velocity decay algorithm balancing upvote momentum with submission age.NoHomepage feeds, tracking actively discussed community topics, trending product discussions.
New'new'Strict reverse-chronological order. Most recent submissions arrive first.NoReal-time alerts, customer support monitoring, immediate response bots, brand mention triggers.
Top'top'Pure net score ranking (upvotes minus downvotes) within a designated time window.Yes (hour..all)Content ideation, curated newsletters, identifying evergreen community answers, LLM training.
Rising'rising'Fastest rate of early positive vote velocity relative to current submission volume.NoEarly trend discovery, comment sniping, identifying viral content before it hits the front page.
Controversial'controversial'Posts with roughly equal ratios of upvotes and downvotes with high comment engagement.Yes (hour..all)Brand reputation audits, sentiment polarization studies, competitive debate analysis.
sort-examples.ts
typescript
// 1. Fetch real-time new submissions from r/node
const newPosts = await client.getCommunityPosts({
subreddit: 'node',
sort: 'new',
limit: 25,
});

// 2. Fetch early breakout posts before they hit hot
const risingPosts = await client.getCommunityPosts({
subreddit: 'webdev',
sort: 'rising',
limit: 15,
});
Step 3

Time Filter Reference

When querying sort: 'top' or sort: 'controversial', the time parameter dictates the time window for calculating engagement scores. If omitted, the default is 'day'.

time: 'hour'60 mins

Breaking Events

Flash news, major software release reactions, security alerts, and live conference updates.

time: 'day'24 hours (Default)

Daily Briefings

Morning team digest, daily Slack summaries, and overnight community pulse checks.

time: 'week'7 days

Weekly Newsletters

The gold standard for weekly tech roundups, curated links, and sprint retrospectives.

time: 'month'30 days

Monthly Analytics

Auditing monthly content performance, competitive launches, and major open-source milestones.

time: 'year'365 days

Annual Retrospectives

Year-end awards, annual industry reports, and benchmarking state-of-the-art developments.

time: 'all'All-Time

Evergreen Knowledge

Curating the highest-voted guides of all time, FAQ building, and LLM fine-tuning corpora.

top-monthly.ts
typescript
// Fetch top 50 posts of the past month from r/typescript
const monthlyTop = await client.getCommunityPosts({
subreddit: 'typescript',
sort: 'top',
time: 'month',
limit: 50,
});
Step 4

Full Post Object Reference

Unlike Reddit's convoluted legacy API—which buries essential fields inside deeply nested data.children[].data trees—SubScraper unpacks and normalizes every post into a flat, predictable JSON record.

Field NameTypeDescription
idstringUnique Reddit alphanumeric post identifier (e.g., "1i7k4mn").
titlestringThe full submission headline as authored by the user.
selftextstring | undefinedThe Markdown body text of self-posts (empty string or undefined for external links).
urlstringOutbound URL for link submissions, or permalink to Reddit for self-posts.
scorenumberNet aggregate upvotes (upvotes minus downvotes). Fuzzed slightly by Reddit anti-spam.
numCommentsnumberTotal number of published comments in the discussion thread.
authorstringReddit username of the author (or "[deleted]" if removed).
createdAtstringISO 8601 creation timestamp string (e.g., "2026-03-12T14:32:00.000Z").
flairstring | nullLink flair badge label (e.g. "Showcase", "Question"), or null.
isVideobooleanBoolean flag indicating whether the submission is hosted native Reddit video media.
thumbnailstring | nullDirect image thumbnail preview URL, or null if text-only.
subredditstringClean subreddit name without the 'r/' prefix (e.g., "typescript").
post-payload.json
json
{
"id": "1i7k3xy",
"title": "Announcing TypeScript 5.8: Modern Const Assertions & Faster Checks",
"selftext": "Today we are excited to release TypeScript 5.8 with improved inference...",
"url": "https://devblogs.microsoft.com/typescript/announcing-typescript-5-8/",
"score": 1482,
"numComments": 184,
"author": "typescript_dev",
"createdAt": "2026-03-11T16:20:00.000Z",
"flair": "Official Announcement",
"isVideo": false,
"thumbnail": "https://b.thumbs.redditmedia.com/thumb-preview.png",
"subreddit": "typescript"
}
Step 5

Fetching Multiple Subreddits in Parallel

Real-world applications often need to monitor several communities simultaneously—such as monitoring competitor subreddits or tracking cross-community tech sentiment. Using TypeScript's native Promise.all, you can fire requests concurrently across 5+ subreddits.

Because SubScraper utilizes rotating residential proxies, concurrent calls do not originate from your server's IP, completely preventing bulk rate-limiting lockouts.

fetch-multiple.ts
typescript
import { SubScraperClient } from '@subscraper/sdk';

const client = new SubScraperClient({ apiKey: process.env.SUBSCRAPER_API_KEY! });

// Target list of developer subreddits to monitor
const TARGET_COMMUNITIES = [
'typescript',
'reactjs',
'nextjs',
'node',
'webdev',
];

async function fetchDeveloperFeeds() {
console.log(`Fetching top weekly posts from ${TARGET_COMMUNITIES.length} subreddits in parallel...`);

const startTime = Date.now();

// Fetch all communities concurrently with error isolation
const results = await Promise.all(
TARGET_COMMUNITIES.map(async (sub) => {
try {
const feed = await client.getCommunityPosts({
subreddit: sub,
sort: 'top',
time: 'week',
limit: 10,
});

return {
subreddit: sub,
success: true,
postCount: feed.items?.length ?? 0,
posts: feed.items ?? [],
};
} catch (err) {
console.error(`Failed to query r/${sub}:`, err);
return { subreddit: sub, success: false, postCount: 0, posts: [] };
}
})
);

const duration = Date.now() - startTime;
console.log(`All 5 communities processed in ${duration}ms.`);
return results;
}

await fetchDeveloperFeeds();

Why Error Isolation Matters:

Wrapping each mapped promise with internal try/catch blocks guarantees that if a single community is set to private (HTTP 403) or banned, the other four requests resolve cleanly without causing an unhandled promise rejection.

Step 6

Saving Posts to a JSON File in Node.js

After extracting your posts, persisting them to disk as a JSON artifact is standard practice for dataset creation, local caching, or feeding into semantic search vector stores. Using Node.js's native fs.writeFileSync with JSON.stringify ensures instant file serialization without extra dependencies.

export-to-file.ts
typescript
import fs from 'node:fs';
import path from 'node:path';
import { SubScraperClient } from '@subscraper/sdk';

const client = new SubScraperClient({ apiKey: process.env.SUBSCRAPER_API_KEY! });

async function exportSubredditPosts(
subreddit: string,
destinationPath: string
) {
console.log(`Pulling top posts from r/${subreddit}...`);

const feed = await client.getCommunityPosts({
subreddit,
sort: 'top',
time: 'month',
limit: 50,
});

// Construct envelope with metadata
const filePayload = {
community: subreddit,
extractedAt: new Date().toISOString(),
totalPosts: feed.items?.length ?? 0,
posts: feed.items ?? [],
}

// Ensure destination folder exists
const directory = path.dirname(destinationPath);
if (!fs.existsSync(directory)) {
fs.mkdirSync(directory, { recursive: true });
}

// Write formatted JSON (2 spaces)
fs.writeFileSync(
destinationPath,
JSON.stringify(filePayload, null, 2),
'utf-8'
);

console.log(`Successfully saved ${filePayload.totalPosts} posts to ${destinationPath}`);
}

await exportSubredditPosts('typescript', './data/typescript-top-month.json');

Pretty-Print vs. Minified

Use JSON.stringify(data, null, 2) during local development for easy Git diffing. For high-volume production crawls, omit indentation to save up to 40% file size.

Streamlining LLM Pipelines

Persisted JSON files can be directly ingested into LangChain, LlamaIndex, or OpenAI embeddings loaders without requiring extra parsers or data cleaners.

Step 7

Extracting Subreddit Metadata & Rules

Often, scraping posts alone is only half the battle. If you are building automated submission agents, evaluating market sizes, or building community dashboards, you also need high-level community metadata: subscriber counts, active online member estimates, official descriptions, and community posting rules.

The SubScraper SDK provides the getCommunityDetails method to retrieve complete community intelligence:

community-meta.ts
typescript
import { SubScraperClient } from '@subscraper/sdk';

const client = new SubScraperClient({ apiKey: process.env.SUBSCRAPER_API_KEY! });

// Fetch metadata and community rules for r/learnprogramming
const community = await client.getCommunityDetails({
subreddit: 'learnprogramming',
});

console.log('--- Community Overview ---');
console.log('Title:', community.title);
console.log('Subscribers:', community.memberCount?.toLocaleString());
console.log('Active Online:', community.activeCount?.toLocaleString());
console.log('Description:', community.descriptionText);

Pre-Scrape Verification

Confirm that a subreddit exists and is public before kicking off large multi-page post extraction runs.

AI Agent Guardrails

Feed official rules into an LLM context prompt to ensure generated responses comply with subreddit policies.

Audience Sizing

Filter prospective communities by subscriber thresholds to focus your marketing intelligence on high-impact channels.

Conclusion

Fetching subreddit posts as structured JSON in TypeScript does not require fighting brittle scrapers, dodging Cloudflare bot detections, or paying exorbitant enterprise fees. With SubScraper, a clean 3-line TypeScript call handles authentication, proxy rotation, and payload normalization out of the box.

Whether you are monitoring viral programming discussions in r/typescript, scraping thousands of market insights with Promise.all, or piping normalized Reddit posts directly into AI agents, SubScraper gives you the speed, reliability, and typing modern applications demand.

Ready to build?

Start Fetching Subreddit JSON in 60 Seconds

Get your free API key now. Start with 30 free requests with access to all 14 endpoints, the official TypeScript SDK, and built-in residential proxy rotation.

No credit card requiredInstant key provisioning30 free requests included

Related Tools & API References

Knowledge Base

Frequently Asked Questions

Q1:How do I fetch subreddit posts as JSON without getting blocked or rate-limited?

Direct unauthenticated calls to Reddit's public .json endpoints quickly encounter HTTP 429 Too Many Requests, Cloudflare bot-challenge pages, and browser CORS restrictions. Using SubScraper solves this by routing requests through a managed pool of residential proxies, stripping out anti-bot fingerprinting, and returning structured, typed JSON via a single API token or the @subscraper/sdk package.

Q2:What sort options and time filters can I use when fetching subreddit posts?

SubScraper supports five sorting algorithms: 'hot', 'new', 'top', 'rising', and 'controversial'. When querying the 'top' or 'controversial' feeds, you can specify historical time filters including 'hour', 'day', 'week', 'month', 'year', or 'all'to precisely capture viral discussions from any window in Reddit's history.

Q3:How do I fetch posts from multiple subreddits concurrently in TypeScript?

You can query multiple subreddits in parallel using standard TypeScript async/await patterns such as Promise.all or Promise.allSettled. Because SubScraper distributes requests across a distributed proxy network, concurrent executions will not trigger IP bans or shared rate-limit throttle penalties.