Show HN: A web debugger an ex-Cloudflare team has been working on for 4 years

886 points by thedg ↗ HN
Hey HN, I wanted to show you a product a small team and I have been working on for 4 years. https://jam.dev

It’s called Jam and it prevents product managers (like I used to be) from being able to create vague and un-reproducible bug tickets (like I used to create).

It’s actually really hard as a non-engineer to file useful bug tickets for engineers. Like, sometimes I thought I included a screenshot, but the important information the engineer needed was what was actually right outside the boundary of the screenshot I took. Or I'd write that something "didn't work" but the engineer wasn't sure if I meant that it returned an error or if it was unresponsive. So the engineer would be frustrated, I would be frustrated, and fixing stuff would slow to a halt while we went back and forth to clarify how to repro the issue over async Jira comments.

It’s actually pretty crazy that while so much has changed in how we develop software (heck, we have types in javascript now*), the way we capture and report bugs is just as manual and lossy as it was in the 1990’s. We can run assembly in the browser but there’s still no tooling to help a non-engineer show a bug to an engineer productively.

So that’s what Jam is. Dev tools + video in a link. It’s like a shareable HAR file synced to a video recording of the session. And besides video, you can use it to share an instant replay of a bug that just happened — basically a 30 second playback of the DOM as a video.

We’ve spent a lot of time adding in a ton of niceties, like Jam writes automatic repro steps for you, and Jam’s dev tools use the same keyboard shortcuts you’re used to in Chrome dev tools, and our team’s personal favorite: Jam parses GraphQL responses and pulls out mutation names and errors (which is important because GraphQL uses one endpoint for all requests and always returns a 200, meaning you usually have to sift through every GraphQL request when debugging to find the one you’re looking for)

We’re now 2 years in to the product being live and people have used Jam to fix more than 2 million bugs - which makes me so happy - but there’s still a ton to do. I wanted to open up for discussion here and get your feedback and opinions how can we make it even more valuable for you debugging?

The worst part of the engineering job is debugging and not even being able to repro the issue, it’s not even really engineering, it’s just a communication gap, one that we should be able to solve with tools. So yeah excited to get your feedback and hear your thoughts how we can make debugging just a little less frustrating.

(Jam is free to use forever — there is a paid tier for features real companies would need, but we’re keeping a large free plan forever. We learned to build products at Cloudflare and free tier is in our ethos, both my co-founder and I and about half the team is ex-Cloudflare) and what we loved there is how much great feedback we’d get because the product was mostly free to use. We definitely want to keep that going at Jam.)

By the way, we’re hiring engineers and if this is a problem that excites you, we’d love to chat: jam.dev/careers

239 comments

[ 3.3 ms ] story [ 253 ms ] thread
*engineers I work with are telling me that some of you are going to argue that types don’t exist in real javascript and that web assembly isn’t assembly
Well it's bytecode, so if you consider java or dot net bytecode assembly then it's assembly, if not then not. If we want to be pedantic, assembly is usually defined as being close to the architecture machine code, which web assembly is not.

Yeah, I fell into that trap, but it's a holiday here.

Haha I knew this would happen!
I'm on holiday so I'll have a go and join the pedantic train. It's not as you say close to the hardware so it's intermediate code. And nobody argues .Net IL is assembly.
A lot of people get confused by assembly in WASM unfortunately, which leads them to claim that it is performant and if it's not, then fixing it is a trivial exercise of just making a fast WASM JIT compiler. Where did I hear this before?

They really should have gone for JSVM IR or something, but that would sell it to no one.

Nice looking landing page. Small bug, when I scroll down the page and then click "pricing" it doesn't take me to the right spot on the page until I click again. On Firefox 125.0.3
On it! Thanks for letting me know!
They dont even support Firefox with thier browser extension product. Looks like this team give Firefox very little love :(
We've started using this. It's fantastic! Great work. Thank you :)
Ah that makes me so happy to hear!! Thank you. If ever you have feature requests or anything, I’d love to hear: dani@jam.dev
A little under 10 years ago I used a product that was designed for improving the website by providing heat maps, but also let you drill down to individual sessions. You could see everything, from console to UI to mouse movements. It was actually pretty neat and incredibly useful for debugging.

It was so neat and useful that as an engineering team we had to hide it from the rest of the company because we were worried it would be used to “spy” on employees. (Id like to note that it was only ever installed on internal tooling that was custom built)

How has jam addressed this concern?

Also, is there self hosting?

Very cool; I use Yandex Metrica for this on public sites
That sounds super useful!

Rather than passively record sessions like the tool you’re describing, Jam is an active model. It lets you actively choose when to record the screen. That way we’re able to avoid any creepiness!

We actually get a lot of requests for self hosting and it’s something I think would be great to add in the future.

I think you should expand the bug report with classic fields like Preconditions, Expected and Actual Result...
Yes! And we wonder also whether LLMs will get good enough soon to reliably auto-fill those. That would be amazing.
"network tab" isn't it a security concern to record API calls?

Also how future proof it is with Google changing Chrome addons permissions / API etc...?

Great concern!

We do two things to address that:

The first is nothing gets recorded and saved (or even sent to our backend) until the user actively chooses to log a bug. So it’s like actively choosing to save a HAR file when there’s a bug.

The second is we scrub all network requests on the client side for anything that resembles PII or sensitive data. We just don’t want it on our servers.

Is there anything else you think we should do?

Definitely building on the Chrome platform there’s always API risk of something changing but I hope Chrome and other browsers will keep it possible to provide a better debugging experience through extensions like ours for people who want it.

Some kind of end-to-end team encryption would be great, I could generate a local passphrase using 1Password or something and put it in the team vault. Then you wouldn't have access to the request metadata (maybe outside of hostnames?) on your servers. Alternatively, an option to simply download local replays and then a desktop/electron viewing client instead of having to upload them to a server would be great too
I like that idea a lot, we should do that
Maybe scrub cookies and authorization headers by default, but make them an opt-in for bugs where they're relevant?

Edit: it looks like this is already the case!

> but make them an opt-in for bugs where they're relevant?

The tool seems to be built for people who don't know what to include in a proper bug report (As they are maybe not technical enough), so not sure how helpful that choice would be for them.

This might be a feature fit for the pro tiers. I would really like it for our QA testers who'd want to include that information but still aren't that technical to know all the places to look for that information.
I see in the Security page [0] that you filter out headers that you think are PII or tokens. And I see that you're willing to receive feedback via email. I don't think this approach is scalable, for the main reason that sometimes we use custom headers to pass tokens.

My suggestion is to have a setting that lists default headers you think should be obfuscated, and the user/team can remove and add to them as they like.

0. https://jam.dev/docs/product-features/dev-tools/security

[disclosure: I work at Jam]

Yes! I think that's a great idea.

(comment deleted)
I can't see anything on jam.dev/careers about where in the world you are located, or if you are open to remote applicants.
Hi! I work at Jam. We are 100% remote.
So anywhere that isn't on the US sanctions list?
Looks very useful for developers but I am not a fan of being on a website that has a keylogger installed onto it. I know this is not the first tool to do this, but the thought of every single thing I do getting stored by some third party tool is a bit scary.
[disclosure: I work at Jam]

I generally agree with you. I think there's a few important things to note:

- Since Jam is a browser extension, its functionality only applies to the user that installed it (i.e., only you, not everyone that visits website X) - All captured data stays locally in your browser until _you choose_ to submit a bug and its details* - You can choose which sites the extension is enabled on if you want to further limit the scope of functionality - We have some user preferences to enable/disable certain functionality of Jam already, and I'm confident we'll add more over time!

* to be fully transparent: we _do_ collect application telemetry so we can identify bugs and performance issues, but we take great care in reporting information that is centered around what our application is doing, not what it is capturing.

JUST in time for me man thank you. I'm going to try it.

Our company has just grown to where it isn't just some engineers and a few sales guys. Now we're trying to keep engineers working efficiently. / accurately, and the support team is ramping up. Making it easy for the sales or support guys to log a good ticket or bug is a never ending challenge. This looks like a great option.

I worked in support for ages and later moved to an engineering role, it's a mountain to climb to get good communication / reproducing things accurately enough for the engineers to really take action effectively.

That is really awesome to hear. Thank you for such a nice message!

+1 –– when I worked at Cloudflare, the support team had to teach all of us how to export HAR files from our browsers for engineers. It's definitely a mountain to climb to get good bug reporting working internally.

(comment deleted)
Nice work, looks like you've got some big names using it too, congrats.

I have some feedback on your homepage & messaging.

Initially scrolling the home page I thought it was a tool for viewing customer bug sessions, similar to TrackJS or Sentry.

Reading this post it became clear this is for product managers and/or team members to file bugs. The line "Built for everyone" on the home page threw me off.

I think if you make the distinction that this is an internal tool for teams vs. an external tool for tracking customer issues it might help make this messaging clearer.

I'm not sure of your plans (perhaps you also intend for external users to use it) this was just my first pass impression.

I think this is a great tool. The structured format combined with the information provided by the extension are what make it valuable to me. The last minute feature is also nice as re-creating bugs can be a PITA.

As a dev, I would ideally also want to be able to view stack traces with source maps, but having a consistent structured format combined with the extension data is enough of a value prop to start using it.

I agree with your comment on messaging, that is fantastic feedback–– reading through the comments here in the HN thread, it seems like that's a common understanding from our landing page. I appreciate you flagging that, we should find better wording to make it more clear.

And that's really nice to hear, thank you so much! Stack traces with source maps. I agree. We integrate now with logging tools like Sentry and I wonder if we can just use sourcemaps from that.

Built for product managers who want quick bug fixes and engineers who want clear and reproducible bug reports.

It's NOT built for everyone.

Right?

I got the same impression at first. Then I saw the drawing tools and I thought that’s a lot to ask from end users. Messaging is confusing for sure.
Landingpage tip: Show your real product at least once, in a screenshot, in a video. Currently all images, videos are stylized which makes me think you are hidding something or at least I do not know what I will get.
That's good feedback!
Looks similar to replay.io
What I don't like about replay.io is that it requires you to use their custom browser.
Hi, I work at Replay.io. There's a very good reason for that "custom browser" requirement.

Session replay tools like LogRocket or Jam do capture a lot of information, but they can only capture what's _in_ the page, and are thus limited by the JS execution environment and permissions. This is still very useful for seeing what the user did in the page, and you do get a good amount of detail (video, DOM, network).

Replay.io works by capturing the browser's calls into the operating system. (This is really complicated! Our fork of Chromium has thousands of lines of custom C++ and JS modifications in order to capture the runtime information and make it queryable.) That enables actually _debugging_ any line of code that ran at any time. That's something that session replay tools _can't_ do.

So, yes, both Replay.io and session replay tools let you _see_ what happened, but only Replay.io lets you _debug_ the code _as it ran originally_. And that's only possible because we do capture the _entire_ browser's execution.

We've got some sections in our docs that dive into this in more detail:

- https://docs.replay.io/time-travel-intro/what-is-time-travel...

- https://blog.replay.io/how-replay-works

- https://docs.replay.io/comparison/session-replay

Not only does Replay.io let you _debug_ recordings of bugs, we've got a Test Suites dashboard that lets you record Playwright or Cypress E2E tests _as they ran in CI_. This is possible because we can run your E2E tests with our own browser, and thus record them as they're running.

Finally, a sneak peak: we're currently prototyping some new advanced functionality that would actually _diff_ passing and failing E2E tests to figure out where a failing test went wrong, surface that info to developers, and help them identify common failure reasons in their tests ("27% of your failures in the last month were due to Apollo Client failing to connect"). Still very early, but we've got the core functionality working! Again, this is only possible because we've recorded the _entire_ browser's execution, and can then replay that browser at will and query for every bit of JS execution that happened.

IDK.

Sanitizing chrome for testing makes a lot of sense as different versions of chrome have been proven to be unreliable.

Playwright uses a sanitized chrome version.

And Cypress does not, and allows users to use their own browsers which may be a reason for some not insignificant portion of flake. They definitely had multiple instances where a particular chrome version was unstable leading to lots of customer complaints regarding their reliability.

honestly that ends up being a feature rather than a bug for me, given how many chrome windows and chrome profiles going on, having it be a separate so entirely is a good thing in my book. I can understand others have a different opinion though. Given all the attention paid to the developer experience, what's weird is CORS not being called out in their browser.
I wouldn't use "cloudflare" as a selling point. People that have used cloudflare know how good the engineering work they do actually is.
Afaik, only Cloudflare's "checking if the site connection is secure" pages are rubbish. And while those are really rubbish (as a web user), the stuff they talk about on their devblogs seems quite solid. I don't think your generalisation is valid.
I think SAML should be available for business tier as well.
+1, great feedback
Is there any reason 'enterprise' features can't be purchased a-la-carte if your org is <X size? Or for a team of X people in a bigger org? Like, if our company has 10,000 people, but only 15 people are ever going to use your tool, we cannot justify the Enterprise pricing... I run into this all the time :(
I'll draft a PR to put them on the SSO wall of shame at https://sso.tax/ when I'm at a laptop
Do you have a similar wall of fame for companies that _don't_ charge extra from SSO support?
To be clear, it’s not mine, but that’s a good idea
This is very cool! One of the hardest things in tech is to teach non tech people to write decent bugs. Our team wouldn’t be able to use this because HAR records at L7 only, after browser content-decoding. A lot of the problems we find end up with symptoms revealed by the encoder or decoder.
Are there extensions planned for other browsers? It seems right now when you click on "get jam" button, it takes me to the chrome web store.
I wanted to know the same and had to dig into the Docs section on the website to find that it currently supports all Chromium based browsers. Support for other browsers is said to be coming in the future.
Yeah! Right now Jam works on Chrome, Brave, Edge, Arc and Opera. We need to build a Safari and Firefox version. It’s a must do!
Too cheap.

It seems extremely inexpensive.

Even though it's not a direct comparison, look at https://www.hotjar.com/pricing/ (and switch to the "pay by month" pricing view)

"Suspiciously cheap" is on-brand for an ex-Cloudflare team
I would have easily paid for any of their best in class DNS, DDoS protection or CDN, but I get all of those free unmetered. Generally when this happens, I get suspicious that they are just trying to kill the competition and then increase price, but cloudflare has been free for almost like forever.
It's no secret that those products are free* with a big asterisk, if you hit some unspecified combination of total bandwidth usage, attracting DDoSes, serving the wrong types of files or moving too much data in regions where bandwidth is expensive then you get an email from sales asking you to pay up or leave. Even if you're on a paid plan, beyond a certain (also unspecified) amount of usage they will ask you to pay more, until you end up on an Enterprise plan where you pay-per-GB just like every other CDN.

They are currently quite generous with how much you can use for free or cheap, but the limits being so vague means they are subject to change without any transparency, potentially pulling the rug on you at any time. That's what I mean by suspiciously cheap.

Do you have any source? I was initially suspicious of the same but found few content in reddit/other forums where multiple users said they have free plan and get millions of requests per day[1]. And I couldn't find a consistent reason for all the sites that cloudflare banned.

[1]: https://www.reddit.com/r/programming/comments/lfa1il/why_you...

Cloudflare does not have any bandwidth limits or request limits in any way. They generally do monitor high usage users on lower plans to try and upsell them to enterprise but you dont ever have to actually upgrade.

The reddit post you linked seems to not really have any content anymore, but I would be willing to bet they broke TOS given you are only really supposed to serve html content over CDN when on lower level plans when you are not using the developer platform to host the content such as videos/images/etc.

Fwiw though, if you are a business you probably should at the very least upgrade to business for the SLA and to create support tickets if needed.

edit: figured out why the article doesnt exist the OP removed it because it was an issue on his end not Cloudflares. (Using bot fight mode on a route supposed to be used by bots) https://news.ycombinator.com/item?id=26072153

I wouldn't call charging $5/user "cheap" when a user might cost a few cents. Pure software services are supposed to have insane COGS. The free tier may eventually get reduced, but with something like this you have basically no lock-in, so I wouldn't consider that a meaningful risk.
Looks like the Jam team have some great investors who will steer them correctly but the founders need to consider whether they can raise the amounts needed the same way Matthew and Michelle (founders of CloudFlare) could back in the day - which enabled the generous CF freemium model and turned the company into the $25bn company it is today.

From a funding perspective things are very different than when CF started.

Cant really compare these at all. Hotjar is for tracking heatmaps and recordings from your regular site visitors.

Jam is for your employees to download an extension and manually submit the recordings when they have issues and is a pay per employee based model.

Jam will generally have very low volume compared to Hotjar which is where the price difference really is

I got really excited before I saw there was no firefox support. Thats a deal breaker for my team :(

I really like this idea and if Firefox support was added I would purchase it in a heartbeat.

[I work at Jam] We do have an internal build for Firefox.. it's coming :)
Awesome, I will check back regularly.

This looks like one of those tools that is genuinely really useful. I would love be able to tell my users to just fireup a browser extension and record a bug, rather than having to step them through how to capture all that data manually.

Well Done!

Bookmarking this page for the futuere: https://jam.dev/docs/downloads-and-browsers/browser-support

That’s so amazing to hear, thanks for the kind words
(comment deleted)
How about Safari? This would be very useful as not every developer has Apple hardware
Orion could make this easier - basically a Safari fork with Chrome / FF plugin support (use it as my daily driver, pretty good and uBlock Origin in Safari? Yes please!)
Consumer facing stuff so nobody in my use case is running much other than a few versions of Safari or Chrome on desktop or really Safari on mobile. (87-92% iOS on mobile)
Yeah sure but this is tool for internal use
Agree, it's a debugging tool for what users see, and users see your app on WebKit in Safari.

So agree, the PdM should not be using Chrome.

I suggested the Kagi Orion path elsewhere in this comment page too, since that would give the PdM extensions, while still seeing what users see.

Man. It will literally never stop being funny to me when people try guilt-tripping developers into adding Safari support. It's the exact same shtick, every single time:

"Can we have Firefox support? It would really help promote the Open Web."

"Sure, it was on our roadmap already."

"AKTLY You need to rewrite your app for Safari support and WebKit coverage because if you ignore that then imagine how bad the iOS experience will be!!! People will blame you instead of themselves because they're simply too domesticated to fix technology issues, so do their work for them!"

...rinse and repeat for everything Apple refuses to support themselves. We wouldn't be begging people for Safari support if it was a successful browser or if people actually cared about it in the first place.

Millions of people use Safari all day every day.

Developers might not like it but people use it. Personally I use it for almost everything now.

It would be great to have a FAQ page on your site mentioning this. I was on your docs page and did a search for "firefox" as well as "safari" in the search box and there were no hits at all. Seems like something that people would often be checking for.
Great feedback, that would definitely be a good thing to add!
Love this tool, want to buy it for our team. Unfortunately, a tool that doesn't support Safari can't readily be used to help iterate products that use default browser or webkit webviews on MacOS or iOS.

We've noticed many tool devs think everyone uses Chrome (because those devs got non tech people in their social circle to install Chrome so "everyone they know" uses Chrome), but users who don't know devs just use whatever's there, and that means iPad, iPhone, and most Mac "end users" are Safari.

Along with being needed for those users, any tool that looks this good and works this magically is also going to appeal to dev teams that can appreciate why users like iPhones and designers like Macs, those teams will want this to work in Safari too.

That said, if you couldn't readily support Safari, I wonder if your extension could be adapted to work in Kagi's Orion browser that supports Chrome and Firefox extensions (to an extent), while being based on WebKit. Perhaps that's a path that could allow webkit debugging.

We would definitely pay for a Safari extension to be installed from Apple app store for each of our UX devs, product managers, and canary users.

For enterprise deployment, it needs to come from Apple app store or be distributed as an installable package through an automation mechanism. MAS (Mac app store) is easiest.

The extension needs to either be free, or have a "full price" version, so that it can be licensed per seat/user through corporate volume purchasing to run on employee devices. The way managed devices work, there's no way for companies to pay for employees to use in-app purchases (IAP).

That’s good to hear. I haven’t used Chrome since Firebug was released.
Professionally I do all of my development in Firefox and only occasionally check that Chrome works
It seems pretty bold move when you consider that V8-based browsers covers 75.5% of the market [1].

[1]: https://gs.statcounter.com/browser-market-share

How is it a bold move? If it works in Firefox it’s much more likely to work in other browsers, whereas if it works in Chrome it might or might not work elsewhere. I switched to Firefox as my default for development years ago because I got fed up of wasting time on issues because something would work in Chrome but not work in one or more other browsers. Using Firefox saves me time and effort so there’s nothing bold about it, purely pragmatic.
I haven’t caused any Chrome issues with this as far as I know. And I’ve built apps with hundreds of thousands of MAUs.
Is it something similar to Userback?
Haven't heard of this before, seems very useful!

God I wish there was something like this for mobile apps, though. Or a website running on the user's mobile device browser. I know Firefox Android does support extensions, but most people on Android use Chrome. And I think on iOS the browser extension situation was miserable the last time I read about it. Either way, an app would probably be the best way to do it.

Yeah! Looks like iOS safari might be the only mobile browser which supports extensions at the moment which is too bad, because the bug reporting problem is very real for mobile
Firefox on Android too, though probably ~nobody uses that.
I do!
Me too, but I'm comfortable being in the ~nobody club. Basically everyone uses the browser bundled with their phone, right? Whether that's Safari or Chrome or whatever Samsung ships these days.
This looks incredible. Congrats on launching this.
Thank you so much!! So kind of you
nice work, love the landing page design, messaging/copy could be improved slightly. I was confused on how it was supposed to help me without going deeper if that makes sense.
Thank you! That’s great feedback, a few other people mentioned the copy could be clearer in the HN thread. That’s so important for us to know, appreciate you saying so!
I wish there were a way to trigger the bug reporter right on their landing page, so I can try it out from a user's point of view.

Edit: never mind - I see this is a browser extension, so kind of a non-starter.

The thing I love the most about Sentry/Bugsnag/Honeycomb/etc is that it is essentially automated observability. That is, I don't need to do a huge amount of integration to have, basically, a prioritized list of bugs along with severity and overall application health.

I like the idea of the browser extension being able to offer these kinds of tools where you wouldn't need to install a heavy Javascript snippet -- but I do think that taking the manual steps out of debugging wherever you can is the most efficient. With Bugsnag et al, I don't need to wait for a user to contact me. I shouldn't have to wait for someone to submit a Jam report in order to see how healthy my app is.

I see these tools as complimentary. Sometimes Sentry &al don't capture a user session (blocking, didn't consent &c) and this tool acts as a nice fallback you can instruct the user to use rather than having to explain how to open dev tools and screenshot the console.
I think they serve different purposes.

Sentry/Bugsnag etc track code errors and exception. But sometimes for an user an error is not a code exception, but the software having the wrong behaviour.

Some examples:

> "The button brings me to the wrong page"

> "I filled out the whole form but the submit button is not enabled"

> "The video should autoplay but is stuck loading"

Those things do not show up in Sentry logs.

Sentry has session replay that should catch all of these things - it doesn't look any higher friction than this tool.
Only if an exception is raised. The alternative is to record every user session, resulting in a privacy mess.

This tool allows the reporter to select when data should be collected