Programmatic SEO is the practice of creating landing pages at scale to capture long tail search traffic. The solution works especially well if your business deals with lots of data, since you can create unique pages that also update. At Dataherald we tried to build a programmatic SEO for our own data and found it to be far harder than anticipated so we wanted to share our experience.
Why was it hard?
- Unless you are AirBnB or Zillow, you will have a limited crawl budget. To get the most out of your budget with a high number of pages, you need to statically generate every single page in advance. Next.js (Vercel) and other static site generators make this easier than it was 5 years ago, but it is still quite a context switch if your entire front-end is a single page application.
- You can’t just drop a table, graph or a few lines of text on the page. Google will penalize your domain for creating spam, low quality content and your domain will be in the dog-house (bye bye organic traffic). The content has to be high quality and engaging for the visitor.
- If your data points will change over time (i.e the number of pages themselves are dynamic) you will have to automate your sitemap creation and maintenance
- You need data. This was the easiest part for us since collecting and organizing external data was our core business, but building and maintaining data pipelines is not easy in general.
How did we solve it?
- We already have millions of data points from pubic and private sources in our warehouse, so data availability wasn’t an issue
- Vercel on the front-end
- We took model driven approach to the backend architecture to allow us to create various front-end templates (HTML code displaying visuals, text, CTAs, etc) from our various data sources. The templates are JSON schema that define how to generate thousands or millions of landing pages. A service creates the pages and then updates them automatically on data source updates
- We are exploring various structured data to text AI solutions but still doing lots of manual templated text writing with SQL.
- Classic SEO optimizations on the page (Ahrefs research, meta tags, H1 tags, canonical, …)
What do you think?
- If the approach above sounds genius or stupid, we would love to hear from you
- If you know of a good structured data to text AI solution, email us
- If you want to implement programmatic SEO please reach out
1 comment
[ 0.25 ms ] story [ 13.7 ms ] threadWhy was it hard?
- Unless you are AirBnB or Zillow, you will have a limited crawl budget. To get the most out of your budget with a high number of pages, you need to statically generate every single page in advance. Next.js (Vercel) and other static site generators make this easier than it was 5 years ago, but it is still quite a context switch if your entire front-end is a single page application.
- You can’t just drop a table, graph or a few lines of text on the page. Google will penalize your domain for creating spam, low quality content and your domain will be in the dog-house (bye bye organic traffic). The content has to be high quality and engaging for the visitor.
- If your data points will change over time (i.e the number of pages themselves are dynamic) you will have to automate your sitemap creation and maintenance
- You need data. This was the easiest part for us since collecting and organizing external data was our core business, but building and maintaining data pipelines is not easy in general.
How did we solve it?
- We already have millions of data points from pubic and private sources in our warehouse, so data availability wasn’t an issue
- Vercel on the front-end
- We took model driven approach to the backend architecture to allow us to create various front-end templates (HTML code displaying visuals, text, CTAs, etc) from our various data sources. The templates are JSON schema that define how to generate thousands or millions of landing pages. A service creates the pages and then updates them automatically on data source updates
- We are exploring various structured data to text AI solutions but still doing lots of manual templated text writing with SQL.
- Classic SEO optimizations on the page (Ahrefs research, meta tags, H1 tags, canonical, …)
What do you think?
- If the approach above sounds genius or stupid, we would love to hear from you
- If you know of a good structured data to text AI solution, email us
- If you want to implement programmatic SEO please reach out