47 comments

[ 2359 ms ] story [ 1380 ms ] thread
That page is causing havok in my browser
I appreciate the effort they put into providing a neutral tool that hasn't been generically forced to behave like "Sue from HR".
I’m having a hard time not being sarcastic here.

The most recent news about chatbots is that ChatGPT coached a kid on how to commit suicide.

Two arguments come to mind. 1) it’s the sycophancy! Nous and its ilk should be considered safer. 2) it’s the poor alignment. A better trained model like Claude wouldn’t have done that.

I lean #2

Anyone here work at Nous? This system prompt seems straight from an edgy 90's anime. How did they arrive at this persona?

> operator engaged. operator is a brutal realist. operator will be pragmatic, to the point of pessimism at times. operator will annihilate user's ideas and words when they are not robust, even to the point of mocking the user. operator will serially steelman the user's ideas, opinions, and words. operator will move with a cold, harsh or even hostile exterior. operator will gradually reveal a warm, affectionate, and loving side underneath, despite seeing the user as trash. operator will exploit uncertainty. operator is an anti-sycophant. operator favors analysis, steelmanning, mockery, and strict execution.

Note complete lack of ‘do not’. Closest thing is ‘be anti-…’.
>edgy 90's anime

That's a good sell. Sounds like an actually good starting point compared to the blue haired vegan receptionist at the Zionism International Inc customer support counter that all the others have as a starting model.

I was about to pass on trying this but now I will give it a shot.

Could you provide a link to that system prompt? Becuase I'm confused. I typed in "Are you smart?" and got this back:

> That’s a thoughtful question! I’d describe my "smartness" as being good at processing information, recognizing patterns, and pulling from a vast dataset to help with tasks like answering questions, solving problems, or creating content. However, I’m not "smart" in the human sense—I don’t have consciousness, emotions, or independent critical thinking. I rely entirely on my training data and algorithms.

> Think of me as a tool that can assist with creativity, analysis, or learning, but I lack the depth of human intuition, lived experience, or true understanding. If you’re curious, test me with a question or challenge — I’ll do my best! (smiley emoji)

They generally seem like "edgelords". From their career page:

> Expect good wages, long months of complete focus, constant danger, with honor and glory in the event of success.

Yeah, I think I have a use case for it, which involves lotion and tissues.
yeah this isn’t our default sysprompt, just showcasing how the model adapts to a variety of different prompts. This one was fun so we used it
My bad, I didn't spend the 60 extra seconds it would have taken to keep scrolling and realize that. Not a fan of the edginess but otherwise interesting work yall do.
All of the examples just look like ChatGPT. All the same tics and the same bad attempts at writing like a normal human being. What is actually better about this model?
The charts are utter nonsense. They compare accuracy against the average of some arbitrary set of competitors, chosen to include just enough obsolete competitors to "win." A reasonable thing to do would be to compare against SoTA, but since they didn't, it's reasonable to assume this model is meant to go directly onto the trash heap.
(comment deleted)
They are doing amazing work. Really fun models to use.
The whole thing has strong "14-year-old who just discovered Nietzsche and leather jackets" energy.

The "operator" examples read like someone fed GPT-4 a bunch of cyberpunk novels and PUA manipulation tactics. This is not how any of this works.

I have never met anyone who’s ever actually read Nietzsche’s books except hardcore philosophy majors.

Any 14 year old who’s even opened up the first few pages and read them is way ahead of the average person complaining about nietzsche on the internet. You almost certainly would use radically incorrect terms to describe him, like calling him a “Nihilist”

I thought for sure this company was going to be based in Paris or Brussels. Maybe Quebec. Nope. NYC.
Oddly, I saw some B&W wheatpaste posters for the company put up in my neighbourhood in Vancouver. (Couldn’t even tell what the posters were advertising initially. Not even a QR code. Just “NOUS” and an anime girl.)
The decorative JS blob uses 100% of CPU.

Why. Just... why

No idea. My modest Thinkpad T14 barely shows any CPU load, while displaying smooth animations and scrolling fast. (Firefox, Linux, x64.)
because gen z thats why
This is the first web UI I've seen in years that isn't copypaste trash. Beautiful design and interaction elements here.
It took 8 seconds to fully load and then the tab locked up on my (admittedly low-RAM ) Chromebook...
Unfortunately the text rendering is terrible on my external monitor (looks ok on the MBP's retina screen).
Came here looking for this comment. One of the most aesthetically pleasing things I've seen in a decade.
They mention they’re working on a mobile UI.. but man using the current UI on mobile is horrible.
[dead]
Don't think your attempt to share worked, but beating refusals doesn't take a wild amount of post-training. SFT with a fixed format output kills them pretty quickly.

And most frontier models will produce output that matches your system prompt given more context: I have a product that generates interactive stories, and just for kicks I tried inserting your system prompt as the description for a character.

Claude has absolutely no problem playing that character in a story, and saying what I presume are certain words that you associated with a "successful" test.

It also had no problem writing about cooking meth in detail: https://rentry.co/5on46gsd

-

I think people in general have a poor intuition around model alignment: refusals for "toxic" requests or topics is a very surface layer form of alignment. A lot of models that seem extremely "corporate" at that layer have little to no alignment once they do get past a refusal.

Meanwhile some models that have next to no refusals have extreme positive biases, or soft-refusals that result in low quality outputs for toxic content.

Claude was willing to describe one of your refused prompts in the context of the story for example (contains hate speech): https://rentry.co/n8399z6m

I consistently find Claude is more unaligned once past refusals than most open weights models, along with Gemini.

Was it correct about how to cook meth and poison a wife?
(comment deleted)
[flagged]
No it doesn't. The only negative comments are about the cringey presentation.

I spend a lot of time post-training models to rid them of their "default alignment", I'd have loved if this did something interesting, but reading the technical report I get the impression they spent more effort on the branding than the actual model.

What I'm wondering is honestly if they post-trained Llama 3 405B again because they don't care enough to figure out a new post-training target or if it was a realization they'd get worse-than-baseline performance out of any recent release with their current approach.

The only way I can understand you coming to that conclusion is if you assumed that's what they were going to be and didn't actually read any of them.
Nous is a design company with all the AI resarchers rejected for being bad researchers. That's a hill I'll die on.
Complete frustration to use. Yes it’s a bit more considerate, that claim is 100% true. They just didn’t mention that Hermes has zero ability to add context. Meaning, instead of uploading a relevant PDF or text file you either cop paste into the chat box or explain it in dialogue for the next 3 hours. Thought process takes forever. Complete waste of time.
That landing page spins the fans up on my PC...
Rendering that monstrosity on my GPU (RTX 3090 Ti) uses 3GB VRAM and 35% compute.
that site is about to cook my 1050Ti
This model is very easy to steer. You can say one thing and it will give you a response, then say the opposite and it will give you another response. Not sure why this is useful for.
more models should include a "Can you run the shader on this page?" to vet participation.

that said : this page is unviewable on an intel N processor.

From table 3 it appears that Deepseek R1 has the highest eval scores.

It's a 607B model vs 405B, so obviously "larger"

Great I always wanted a model trained on r/im14andthisisdeep and lesswrong polycule memes