Hey everyone, Josh here the creator. As a developer, I’m always looking out for an emerging market trend on which to bootstrap my own SaaS product. This is since market awareness and timing are often so critical, on top of execution skill of course!
So I built Trennd. Under the hood it continually monitors the web for interesting keywords & topics, classifies them using Google Trends data and packages everything as a neat web app where other people can contribute too.
The app itself is built with the Next.js React framework along with Express, Bootstrap and MongoDB. Next.js was new to me, but made sense since it comes with so much out of the box, including server side rendering.
Let me know if you have any questions, and any feedback is much appreciated!
Hi, it would be nice if the keywords had short explanations. I had to find out what an affinity designer and an enneagram was. Bubble tea was also quite new.
Of course - I have scripts constantly monitoring the web for potentially interesting keywords. I then check google trends data for that topic/keyword and analyze the gradient of the line of best fit over different timeframes. If the gradient is higher than a certain (arbitrary) amount, then I class it as a trend!
Hey Josh, very interesting concept. Thanks for building it!
I tried to use the "technology" category, but the results are not what I expected. Perhaps it would be more useful if I could narrow it down to more specific things, e.g. programming languages, or databases.
> I have scripts that are crawling the web in places like twitter, reddit and facebook to generate keyword ideas.
So the trends that it detects is biased towards the corners of the web that I am monitoring. Which skew heavily tech!
Awesome I enjoyed making some trend charts, and the results are fun (i.e I added DevOps, Erlang, Elixir). I think there is a risk it could conceal too many errors and biases for it to be useful for trend analysis. Using services like Twitter and Reddit might introduce biases into your data skewing trends. The challenge is using data and not understanding the intent (is it positive, negative or neutral/sarcastic potentially negative/positive). Google Adwords is good at this. I applaud your effort; I hope you keep going but its a tough nut to crack. I use markets like Google Adwords Tools to do this at the moment, have you looked at it? I think more info around time, intent and volume.
Pretty neat, I see the most granular time scale currently supported is a 3-month window -- are there any plans to increase granularity towards a (near) real-time system? I get your crawler might be rate-limited at that scale and you would likely have to rethink the entire system architecture, but I think real-time data would open up a whole new dimension of possibilities for detecting/predicting trends perhaps even before public awareness catches up. I'd imagine there would be parties interested in paying for that kind of capability if that's the direction you're planning on taking this idea
I'd love to go as granular as possible, at least 30 days. Real-time would be ideal, but as you note - Google Trends rate limiting soon becomes a hurdle especially when you potentially multiply by different geographies.
Yes - City would be great for people doing stuff around activities, events and local places. But Country would be more powerful for e-commerce I suspect (my usecase).
That's very neat and interesting; I guess the two questions are
- how do you generate keywords to search for ?
- how do you filter out noise (or perhaps you don't filter it) ?
Yeah they're normalized relative to a scale of 100, as it's Google trends data. In the future I want to include actual search volume numbers as a line on the chart too though. Just need to figure out where to get that data first!
The examples on the front page looks surprisingly good, which makes me suspect they were hand-picked. Kind of beats the purpose of the app if I'm just shown trends the author and me already know about, since we seem to inhabit the same subcultures.
When building a system that surfaces interesting trends, you have to be careful about curating which historic examples you show. People will craft their understanding of predictive power based on those examples... but if the examples aren't representative (i.e. randomly sampled) of the algorithm, then you're selling people rainbows and unicorns.
It's interesting to see that topics like 'fentanyl', 'opiod' have done up.
Also 'criminal defenses', 'wrongful death claim' have gone up a lot as well - Perhaps they are related to spikes in searches for plastic surgery topics.
Also it's weird why almost all the topics on the front page are related to software development. It's like there is some kind of conspiracy to turn the world's population into software developers.
Right now spike/potential/hyper classifications are my subjective opinion.. which I hate because that varies depending on the day I'm having and my specific biases.
Figuring out how to classify them mathematically is next on the to-do list. For example a spike would have extreme outliers in the middle and hyper would be over a certain exponent.
This is awesome. I'd love to have the ability to view topics based on category. IE. see the biggest trends in entertainment or the biggest trends in sports players, or the biggest trends in startup brands.
This definately has a market, but i'm not sure SaaS is it. More typical eCommerce products have a "flavor of the day" like effect, where an article is written in some niche community, and everyone in that community rushes out to buy the thing. Then a few months later, the craze is over. If you're Amazon, you can spot these trends easily (because they have everyone's sales data from the marketplace). If you're a small eCommerce company, unless you just happen to have that product on hand, and someone just happened to have found you, you may never even realize the trend existed. Being alerted that there is a sudden demand for a product, might give you a headstart on marketing and might help you negotiate a deal with a supplier before the supplier realizes what they have.
I'm a moderator of one of the bigger Facebook advertising groups that exist. This caught my eye instantly as the e-commerce and affiliate crowd are going to be all over it. They love being able to catch trends relatively early. This product is ready made built for flavor of the moment marketers.
For sure. So shorter term. They're going to want to move faster than 3 months. Maybe like a past 30 day trend if possible? I would say also if there's a way you could dig into Alibaba and see if there's a bunch of new items of the same type getting listed recently that would be extremely interesting for them as well.
That's an interesting angle regarding Alibaba. The hard part might be automatically grouping similar items, but definitely can see the huge value in this.
When I was doing affiliate marketing a decade ago, what I did manually was watch informercials and jot down the names of all those products. Then manually run searches in Google Trends to see which ones were spiking, and create content for those products (ie reviews, guides, etc). Here's an example of a product that really took off after they did an informercial in 2006: https://trends.google.com/trends/explore?date=all&q=almighty... . I created a page for this product and was ranked #1 instantly. Because nobody else was writing about it.
I stopped doing affiliate marketing so don't know what people do now, but just throwing you some ideas.
not sure i agree so much because its not showing things that are just starting to trend, which is where the real money is at... being in front of the ball rather then behind/beside it. best i can tell this service just shows you the search volume history from about 1100 select terms that anyone can already lookup using googles trend tools.
id be much more interested to see terms each day whose search volume has increased 10x or something... that would be very useful for people trying to get in front of trends.
Appreciate your insight - really trying to figure out who I should be building this for, so I know what features and direction to focus on. I was thinking eCommerce entrepreneurs could be it, but then a lot of other people potentially find it valuable too. I want to see who keeps coming back to Trennd in the future, the power users, and ask them why/what they're using it for!
Thanks Josh, I sent you an email I think you answered most of the stuff here though just came back to the comments now. Would still be great to have a chat sometime
The "flavor of the day" comment really caught my attention, been thinking of a startup between those lines. I think with people having hundreds of decisions to make everyday from a million choices, it would help to answer those questions for the users without them having to spend time deciding.
I mean its great to have review sites, but what should I have for lunch today? what is trending just give me the answer I am happy to pay $5 a month you decide for me. What programming language is trending?
Will look into it at some point, happy to have a chat with anyone who's interested my email is in my profile
Selling the data as an API is definitely a potential. I was originally thinking to allow CSV downloads of the data. Would an API be more useful for your client's use case?
You seem to have done most of the hard work, collecting the data. I'm not sure they'd care what format the data was in. Providing it could be expanded to cover the topics their clients need.
And/or sell the service where you BYO keyword list and this ranks them. Keyword lists can themselves be scraped from forums and websites in the marked the client is interested. The scraping could also be an add on service.
This would lock in nicely with the keyword research that ahrefs, moz, etc. offer. And other various colour hat seo software.
I guess the most logical conclusion is that every time they make a sacrifice under the full moon during the equinox, they get rewarded with more interest in Gmail.
The key value-add, in my mind, from that product, besides the trend, is the editorial content they include in the email. Of course, that's just me as a layperson.
I'd probably dupe their pricing model and approach: i.e., a limited number of weekly/monthly trends, subscribers get more.
Definitely - the glimpse newsletter looks good and is the same concept of surfacing new Google trends. I didn't like being limited to just a few trends per month everyone else sees via newsletter though.
I was looking for an interactive app with 1000 trends I could dig through. But as you say, then the editorial content/insight for each trend is key value-add. That's part of the reason I've built in for users to be able to add their own trends, insights and comments. If we can crowdsource it, then to some extent we don't have to trade-off quantity and quality.
Very cool! As a front-end dev it's interesting (but I guess also not surprising) to see React and Vue with very steep curves.
I wanted to try adding Svelte, so I signed up by email, but that doesn't seem to be working. Then I tried Sign In With Twitter but, wow, it requires an awful lot of permissions! Among them... Follow new people, Update your profile, and Post Tweets for you. I'd be a lot happier signing in with Twitter if it were limited to read-only abilities.
This could have massive potential with music industry A&R's looking to capitalize on viral songs. That's where virtually all recording contract money is going nowadays.
I'd also like to see downward trends, not just growing trend. From the last page or two, I see some downward trend, like "blog", "facebook", and "gmail". I don't know if it means the decline of popularity, but it is definitely interesting to see the declining graph.
119 comments
[ 2.8 ms ] story [ 190 ms ] threadSo I built Trennd. Under the hood it continually monitors the web for interesting keywords & topics, classifies them using Google Trends data and packages everything as a neat web app where other people can contribute too.
The app itself is built with the Next.js React framework along with Express, Bootstrap and MongoDB. Next.js was new to me, but made sense since it comes with so much out of the box, including server side rendering.
Let me know if you have any questions, and any feedback is much appreciated!
Also, if you are choosing the keywords then wouldn't the data be inaccurate for predicting trends as trends occur randomly?
So the trends that it detects is biased towards the corners of the web that I am monitoring. Which skew heavily tech!
It can probably be used for lots of things: startups, as you mentioned, but also journalism ideas, investing ideas, job hunting and many more.
I'm working on something similar for a internal tool, so I'm curious to see what your solution is.
Thanks!
I tried to use the "technology" category, but the results are not what I expected. Perhaps it would be more useful if I could narrow it down to more specific things, e.g. programming languages, or databases.
Keep up the good work!
> built with the Next.js React framework
Wait wait wait... do you mean to say there's now a framework for a framework? Javascriptland continues to confound.
Right now, I'm filtering out the noise and crud manually! So it's a combination of automation and curation by hand.
I actually hadn't considered OSS, which part of it specifically would be most interesting to see?
Right now they're monitoring the obvious techie havens, but I'm looking to expand their coverage to be more comprehensive.
But duly noted! I will make that clearer.
Are the trends on the frontpage sorted by upvotes?
Examples
1) https://trennd.co/trend/ketogenic-diet
2) https://trennd.co/trend/plant-based-diet
No idea what that means, but it suggests why you're not seeing the same data.
Also 'criminal defenses', 'wrongful death claim' have gone up a lot as well - Perhaps they are related to spikes in searches for plastic surgery topics.
Also it's weird why almost all the topics on the front page are related to software development. It's like there is some kind of conspiracy to turn the world's population into software developers.
Figuring out how to classify them mathematically is next on the to-do list. For example a spike would have extreme outliers in the middle and hyper would be over a certain exponent.
https://trennd.co/?categories=%255B%2522sport%2522%255D
Definitely looking to classify with more precise categories in the future like sports players etc as you mention
If I were to build Trennd towards these e-commerce guys, what would you improve/add/change to make them love it?
That's an interesting angle regarding Alibaba. The hard part might be automatically grouping similar items, but definitely can see the huge value in this.
I stopped doing affiliate marketing so don't know what people do now, but just throwing you some ideas.
id be much more interested to see terms each day whose search volume has increased 10x or something... that would be very useful for people trying to get in front of trends.
I mean its great to have review sites, but what should I have for lunch today? what is trending just give me the answer I am happy to pay $5 a month you decide for me. What programming language is trending?
Will look into it at some point, happy to have a chat with anyone who's interested my email is in my profile
I am currently working with a client who is exploring the options for building out a similar system.
This would lock in nicely with the keyword research that ahrefs, moz, etc. offer. And other various colour hat seo software.
Are these all public sources?
Have you considered backtesting it to see if you can get a trading signal out of it?
Now why would that be...
The key value-add, in my mind, from that product, besides the trend, is the editorial content they include in the email. Of course, that's just me as a layperson.
I'd probably dupe their pricing model and approach: i.e., a limited number of weekly/monthly trends, subscribers get more.
I was looking for an interactive app with 1000 trends I could dig through. But as you say, then the editorial content/insight for each trend is key value-add. That's part of the reason I've built in for users to be able to add their own trends, insights and comments. If we can crowdsource it, then to some extent we don't have to trade-off quantity and quality.
I wanted to try adding Svelte, so I signed up by email, but that doesn't seem to be working. Then I tried Sign In With Twitter but, wow, it requires an awful lot of permissions! Among them... Follow new people, Update your profile, and Post Tweets for you. I'd be a lot happier signing in with Twitter if it were limited to read-only abilities.
Upgraded my mongodb atlas cluster to resolve the issue, apologies.