Launch HN: Just words (YC W24) – Optimize your product's copy for user growth
“Copy” in this context means short-form content such as landing page titles or email subject lines. Small and seemingly insignificant changes to these can lead to massive growth gains. We observed this on many occasions while working at Twitter. Tweaking a push notification copy from “Jack tweeted” to “Jack just Tweeted” brought 800K additional users to Twitter.
However, coming up with the copy, testing it, and optimizing different variants across users would take us forever—sometimes months. There was no methodical way to optimize copy on the fly and use what we learned for subsequent iterations. The entire process was managed ad hoc in docs and spreadsheets.
After experiencing this pain at Twitter, we observed the same problem at other companies like Reddit. After years of this, we are convinced that there’s enough evidence for this pain across the industry.
In our experience, the main challenges with copy optimization are:
Engineering effort: Copies are hard-coded, either in code or config files. Every small change requires redeployment of the app. To run an A/B test, engineers must write if/else logic for each variant. As a result, one copy optimization becomes a 2-week project on an engineer’s roadmap, and companies are only able to prioritize a small number of changes a year.
Fragmented content: There is no copy repository, so companies lose track of the history of changes on a particular string, learnings from past experiments, and their occurrences across the product. With no systematic view, product teams make copy changes based on “vibes”. There is no way to fine-tune next iterations based on patterns obtained from previous experiments.
Lack of context: Companies either test 1 copy change at a time for all users, or rotate a pool of copies randomly. In an ideal world, they should be able to present the best copy to different users based on their context.
We built Just Words to solve these problems through 3 main features:
No-code copy experimentation: You can change any copy, A/B test it, and ship it to production with zero code changes. All copy changes made in our UI get published to a dynamic config system that controls the production copy and the experimentation logic. This is a one-time setup with 3 lines of code. Once it’s done, all copy changes and experiments can be done via the UI, without code changes, deploys or app releases.
Nucleus of all product copy: All product copy versioning, experiment learnings, and occurrences across the product are in one place. We are also building integrations to copywriting and experimentation tools like statsig, so the entire workflow from editing to shipping, can be managed and reviewed in one place. By storing all this in one place, we draw patterns across experiments to infer complex learnings over time and assist with future iterations.
Smart copy optimization: We run contextual Bayesian optimization to automatically decide the best-performing copy across many variants. This helps product teams pick the winner in a short amount of time with one experiment, instead of running many sequential A/B tests.
We are opening up our private beta with this launch. Our pricing is straightforward - a 60-day refundable pilot for $2000 (use discount code: CTJW24), for one of the following use cases: landing pages, push notifications, email subject lines, or paid ad text. We will show visible growth gains to give you a substantial ROI on your pilot and refund the amount...
79 comments
[ 199 ms ] story [ 2957 ms ] threadPerhaps it would be more effective to put a lower-quality landing page in as your demo. Off the top of my head, something like https://www.intuit.com/ might work. Their existing tag line is "The global financial technology platform that gives you the power to prosper". Doesn't mean much to me - I'm sure your tool could give me some better options, which would serve much better for a demo.
A good example would be a company that isn't an unavoidable juggernaut in its space and also isn't great at marketing. There are many open-source projects that would work because they don't have a big marketing team, but they might still be well known.
In speaking to SMBs and large companies, our insight was that the problem of copy optimization resonates more with larger companies, as smaller companies are more focused on survival/basic marketing techniques like opening up new channels. Larger companies have already exhausted those levers, and are ready for more sophisticated optimizations.
Did you talk to Stripe? Do they do systemic copy optimization on their landing page, or not?
Sort of. Companies copy changes at larger companies because they're addressing a different audience and different set of needs for that audience.
TL;DR - concrete language / early stage ==>> abstract language / larger enterprise
Yes! Look at the alternatives the tool generated:
* Maximize Revenue with Secure Transaction Processing
* Elevate Digital Commerce with Trusted Payments
* Unlock Opportunities with Secure Transactions
* Simplify Your Online Payments
* Accelerate Your Online Business Growth
These are just... bland. Generic. I've seen a thousand webpages with taglines just like these.
AI can be quite bad at generating unique responses. Its answers are middling. This can be great for, eg, coding where you want a copilot to generate known algorithms and approaches, but terrible for marketing where you want a slice of brilliance and genius.
Would turning up the AI temperature and other settings help, maybe?
A little ironic that this company is all about wording it better :).
Now, if they took a defensive position, trying to justify their non-ideal wording, that would be ironic.
An opportunity to dogfood your product!
Not a gotcha, just genuinely curious.
How are you handling messaging consistency for specific users? Ie. A user is shown an experimental string of copy with one value prop and you want it to show up multi-channel for them. Do you have a way to associate the experiments on a per-user level?
example: website_landing_page_title and email_subject_line can be part of the same experiment for a multi-channel experiment.
For example, we'd have the same opening email sequence, same retargeting, etc. for customers across channels but we'd want to have consistent messaging around offers (ie. "$1000 off when you sign up" vs. "first month free" kind of stuff). It was tricky because we didn't want to advertise the same offer to everyone, so we wanted to carefully segment who was getting which offer and keep it the same over a window of time.
Unfortunately we didn't have a perfect solution. The closest we came to it was to have an experiment ID tied to their user account. Then we had a system where we would define the different experiments (including messaging and promotions) with each experiment having an experiment ID. It was far from perfect but worked.
So some sort of tracking knows that your opti-cop worked well on me for some other site you service, so then it tries a similar style for another site (who uses your service)?
This has been tried and tested a few times in industry, notably at Netflix and Duolingo for artwork and notifications
https://towardsdatascience.com/multi-armed-bandits-part-1-b8...
So why don't they offer that?
I know it's not because the data is proprietary or private, because basically all the information you need is visible on Facebook Ad Library, more than enough to answer most questions about authoring copy by sheer mimicry.
You emphasize the UX here a lot. I don't know. I think Meta's UX is fine, end of story.
This isn't meant to be a takedown. It just seems intellectually dishonest. Anyone who has operated these optimization systems at scale knows that the generalized stuff you are saying isn't true. You're emphasizing a story about engineers versus product managers that is sort of fictional, like the right answer is the one that most companies are already taking which is to not A/B test at all, because it doesn't matter, and when you do see results they are kind of fictional.
And anyway, it belies the biggest issue of all, and this is actually symptomatic of Twitter and why things were so dysfunctional there for so long, long before they were taking private: you are saying the very Twitter esque theory that "Every idea has been had, and it's just a matter of testing each one, one by one, and picking the one that performs best." That was the core of their product and engineering theory, and it couldn't be more wrong. I mean if you don't have any experience or knowledge or opinions, why the hell are you working there?
> However, coming up with the copy, testing it, and optimizing different variants across users would take us forever—sometimes months.
The right answer is right in front of you! Don't do it! Don't optimize variants, it's a colossal waste of time!
HN is a good example of this. Headlines that are too outrageous or catchy do not get upvoted that much here but something simple like “I created a rust debugging toolkit” will likely get upvoted like crazy, while something like “I got laid off a day after I got pregnant. Here’s what happened” probably will get buzz on TikTok.
The big companies are especially susceptible to these distractions because they have the budget to blow chasing micro funnel optimizations. It sounds reasonable, but in my experience i agree it's a waste of resources.
It's too hard to prove causality. Entire orgs are set up to run rigid experimentation analysis, and prove incrementality so we can trust the data. But that should be warning of just how complicated it is. and we can't 100% trust it. There's external factors. hence a button color and a line of text shouldn't make the cut list of priorities. it's not that significant.
if justwords can make this trivial then at least it's minimizing the distraction. that's a win, and fwiw i think b2b wants this product, so the company can do well. i just don't think micro content optimization, after doing it for unicorn for 8 years, really moves the needle like people believe the data shows. People use PMF products in spite of their UX! (for example)
1) Is copy even important? I think it is. If this post was titled, "Auto-tune experimentation for short-form content optimization", it might make half the audience confused about the product. In fact, the 1-liner we use for HN is very different from when we talk to VCs, because the audience is different with different goals & backgrounds. I guess the point I am making is that messaging has to be contextualized, depending on users, platform, and goals.
2) PMF vs copy - I agree that the two are orthogonal. Copy is not going to solve for the lack of a PMF (and it shouldn't). Exactly the point above - the goal is to help more and more users comprehend what you do, hopefully in a way that's more personalized to them.
That's the challenge: conversion funnel is complex with many factors. and largest one of them, in simple terms, is PMF.
if we measure downstream like clicks or inbound leads etc, that's more aligned with "discovery of PMF" and that's good. But it should be stacked ranked as so, it's not driving the needle. it's exploratory.
Wouldn't 'all the data' by definition have the data for various audiences at least calculable?
Which audiences am I optimizing copy for? Where do they come from? Some Google, Meta TikTok or Apple owned channel right?
Google has indexed every website. Meta has every ad. Can't they just tell me what copy to use? Why don't they? I mean, they know! They know what copy works best, for pretty much everything. They can sort by clickthrough rate, revenue due to the purchase data they have, they have everything! You talk about SMB - they know every SMB! They know your margin and your COGS and whatever because they in aggregate they observe rational spending where all the cost is eaten by marketing; they know your potential market, etc. They know all this. They don't need to run tests. They can look at very recent, weeks old, historic data, and they have way more than enough samples to answer these questions to more or less the same degree of certainty and scientific rigor that any SMB doing it themselves as a first party can do.
I mean if they wanted to, they could run the A/B tests for you! Google could "just" serve a different web page with slightly altered copy. And see if more people "click" or "convert" or whatever. They have better technology, 1,000,000x more data... Why don't they do this? You wouldn't even need UX. It could just happen, you would check a box, and they would do this.
> fundamentally separates strings (copies) from code... and lets you update them effortlessly... The ability to make scientific edits without re-deployments and accelerating continuous iterations based on user feedback...
You keep talking about UX for developers and product managers. These are UX things. It doesn't actually matter. The existence or non-existence of what you're talking about doesn't correlate to higher or lower conversions, it isn't a scientific opinion on the practice of optimization, it is just a bunch of UX patterns to achieve it, but it could be achieved in many ways, perhaps with even better UX. Like in the example I gave, where Google "just" does this for you, which is the best UX because there is no UX, you don't need to separate strings from code, and you don't need to update them, because you don't need to do anything. Google could just do this. They own the channel, they see everything, they have the technology.
So why don't Google and Meta and Apple offer an automatic optimization product? You ought to have an opinion, it can't just be, "I don't know." I mean the sort of obvious answer is that "optimization doesn't really work" instead of "three paragraphs of bullshit."
> Curious why you think A/B test results are fictional? Getting stat sig results is probably the surest way to conclude results. Perhaps there's a different angle you are talking about here.
Well one reason I am very confident they are fictional is because the people who own the channels for a decade haven't offered a tool to do this.
I mean maybe they will. Maybe it was a technology problem, but I don't believe so. You could have Markov Chained your way through 5 word long taglines and whatever. They didn't need to way for generative AI to create valid test strings for people's websites. Indeed they could just let you copy the best performing taglines they see in their systems. Why. Don't. They?!
> Given the number of users that get exposed to every change a consumer company as large as Twitter makes
Another POV is that every change they made was bad. They thought they were a product organization, and they are really a backend engineering organization, where the best decisions are all based on first principles or executives' opinions, not on some unknowable measurement about audiences.
The pitch would be “automatic ab testing for your form submissions”. I talked to a few local lead gen companies years back and they thought it was a neat idea, I just never got around to building it.
Rather we should optimize for user understanding or satisfaction - whichever fits. These are harder to capture but FAR more beneficial to the consumers of content.
A few examples from industry include: 1. Netflix artwork optimization https://netflixtechblog.com/artwork-personalization-c589f074... 2. Duolingo notifications optimization https://research.duolingo.com/papers/yancey.kdd20.pdf
The things are make us stand out -
1) There is no tool afaik that looks at inferential learning based on past experiment results on copies. The experiment analysis and the continuous feedback loop is missing. 2) To make (1) happen, the first step is copy versioning and management. Without a tool that can abstract strings in a CRM, and monitor learnings over time, it's hard to make (1) possible. 3) We integrate with tier 0 services like notifications with a low latency solution that makes copy iterations possible for in-product features, outside of logged out surfaces like web pages.
Would be interested to know if you have seen software that solves for (1), (2) and (3)?
I am still a little fuzzy on how this might play out on a real site, since lots of monthly users doesn’t always mean lots of conversions (especially for one-time, high dollar transactions).
Let’s say I have a page that gets 100k unique visitors per month. I show them 5 different variants of a “nudge” widget. Some do better than others, but they all hover at <1% CTR.
While there may be a story to tell as far as winners and losers (e.g. mobile users converted to variant A at 2x the rate of desktop users), how do you confidently report a “winner” from what amounts to a few thousand conversions in a month?
In my experience, copy is used the same way as code. And just as you probably wouldn't say, "I changed the codes for the three apps," most people wouldn't say, "I rewrote the copies for the three websites."
Unless you’re - shudder - a data scientist
"Healthcare, infrastructure, K12 education and other essential services are a mess because we lack the talented resources to solve those fundamental problems"
"We largely do have the resources, it's just that they're all trying to get people to click ads, getting people to waste time endless scrolling on Reddit or Twitter or Instagram, managing their brands' social media accounts, etc."
This is not a problem in need of a solution. You both seem like really talented guys, and it's depressing that this is what you've decided will help make the world a better place.
That you're part of the YC24 cohort says a lot about what's wrong with the tech industry's priorities right now
Don't hate the players, hate the game.
If we deregulated there would be a rush of innovation and the next YC cohorts would have many more startups targeting those industries.
>> That you're part of the YC24 cohort says a lot about what's wrong with the tech industry's priorities right now
Couldn't agree more, it seems that there's a big disconnect of what startups are getting founded and what would actually change the future.
Personally this ranks somewhere in-between web 3.0 and generic AI <feature>.
Not claiming you did this, but in your example of "Jack tweeted" vs "Jack just tweeted", it's so often bad statistics or cherry-picking that leads to results like this. Testing null hypothesis is hard, and many people will take a reality like this chart [0] and claim that their A/B test is successful.
I definitely think there's a gap in the market since Google sunset optimize, but $2000 seems kind of steep for something that many people got for free before.
[0] https://i.ritzastatic.com/images/3dcdd822dbd241b1b3ddeeb9540...