I wonder if anyone yet has the job title "Query Engineer"? [EDIT: "Prompt Engineer"?]
Also: if ML tech stalled now I expect we'd grow to have many Query Engineers, but if it doesn't do we still? For how long is this sort of experimentation what getting the best performance out of an AI will look like?
I'm not sure if I believe that we'll have dedicated positions for query engineers because it's so hard to validate. Programming may seem like wizardry, but it's the rule-bound, alchemy version of wizardry, this on the other hand is more like communing with spirits. All the conjecture in this post could be right on the money, or total bunk.
I don't see how anybody could reasonably hire someone for this position under this condition. How would one possibly discern between someone who can truly suss out what's going on and a techpriest who just recites learned dogma? I imagine it will definitely be an important skill, but I imagine that just like 'knowing how to google', it will be something that's just implicitly attached to a normal job.
Some people are better at it than others if you just watch the hashtags on Twitter. There will very clearly be value derived from it before we understand it. It’s very likely, in my opinion, that people will be hired for this type of work.
When GPT-3 came out it was very hard to coax even with prompts. Then OpenAI made Instruct GPT-3 and it suddenly became malleable and much better at taking a hint. I bet prompting is going to become even easier with time, so it won't be a skill one has to learn over years of practice anymore.
That's a very different sort of Query Engineering than what the author of this post is engaging in; I'm trying to talk about the latter (apparently also called Prompt Engineering).
My job is not incredibly far off of this. My company is based on Poland and so all the the software is written by Eastern Europeans with very different ideas of what words mean, how ux works, and how software should be structured. I have ~15 deep learning algorithms that I control some inputs/filters/etc on and my job is to get specific outputs from them every day. But I never know exactly what I’ll get and the raw inputs change so I need to respond by making changes every day. I have slowly gotten better at learning what may happen when I pull a lever, but it’s all feel and guesswork. I have no access to documentation/etc so can’t look inside the black box.
"And when your surpassing creations find the answers you asked for, you can’t understand their analysis and you can’t verify their answers. You have to take their word on faith—Or you use information theory to flatten it for you, to squash the tesseract into two dimensions and the Klein bottle into three, to simplify reality and pray to whatever Gods survived the millennium that your honorable twisting of the truth hasn’t ruptured any of its load-bearing pylons. You hire people like me; the crossbred progeny of profilers and proof assistants and information theorists…
In formal settings you’d call me Synthesist."
As I read it, the entire book is him doing synthesis from the experts to us, the reader. He explicitly calls out that there's one character that he lets "speak for herself", implying that for everyone else, he's always translating, dumbing down for consumption. He actually says that their normal language would be unintelligible to us because it's an opaque multi-language pidgin that relies on realtime consensus access to be parsed.
You are right, that is how it gets described at the beginning but, again, mostly inconsequential flair, no? Like adding Universal Translators and never having a Darmok and Jalad episode.
Yeah, that's a fair accusation. Part of the problem is of course that it's hard to have a novel where you only notice the primary conceit if you don't understand what the narrator is explaining. :-)
Like, since perfect synthesis is undetectable; it just sounds like what you're hearing is straightforward common sense. You only notice the process when it fails. So for sure, the book could have played with that a lot more, especially since Siri's tools are meant to be broken towards the end, when he's implied to be penning this. I think there's bits where the book suggests that's what's going on ("I can't tell you what it means, only what it says"), but it never came across well to me.
But then again, maybe that's just how a failure of synthesis would look. Like, you still want to understand what the book is saying, but if the mechanism is working it's invisible, and if it's not working, then you're still relying on it working to produce the thing you're reading. So it's just inherently hard to shine a light on the thing itself with itself, which of course plays into the central theme as well. Ie. because Siri is implied to not be fully conscious, it's hard for him to catch himself in the act of translating for us readers.
(Of course we may say that a book designed to be hard to read is still hard to read.)
Not completely obvious from that wikipedia section but one important reason he didn't use them is that he died a few years before the first telescope was invented.
Edit: replacing 'Tycho Brahe' with 'Zombie Tycho Brahe' in the prompts could be one effective way to fix this
He also died from being too polite to take a bathroom break.
I think his nose was more or less skin-colored though, so the current pictures are okay. (DALL-E is supposedly programmed not to be able to generate real peoples' faces anyway.)
A woman who competed in a radio station’s contest to see how much water she could drink without going to the bathroom died of water intoxication, the coroner’s office said Saturday.
Water intoxication is a separate issue, an acute electrolyte imbalance, a dilution of blood; probably nothing to do with not peeing but with rapid drinking. If her kidneys had time to shed all the water she would have wet herself and not died.
Look closely. The depicted devices are no telescopes. They would interface to an eye. They are fancy bottles, and he isn't feeding the goose but himself.
Oh, wow! I didn’t fully realize what was weird, but it’s very obvious once pointed out. It seems like “longish mechanical implement close to the face” is so strongly biased towards “bottle” that the AI has difficulty moving the telescope up to the eye level.
Art is using the qualities of a medium to enrich your message. Dall-E and it's training data are a medium with qualities that we can use to enrich a text based message.
It's not creative in the sense that it's clearly an automaton and has no awareness of culture, history, or context.
It seems to operate like a huge multidimensional map of an image space with some verbal tags stuck into it rather randomly, and with style transfer applied as an operator to merge different features in the results.
What it needs is an equivalent map for the concepts behind the signposts, and also a mapping between concepts and images.
The concept map would be very hard to train because a concept space that includes all of culture and history would be huge and messy.
A large concept map with a huge set of associations is one of the features of educated humans. But the concept mesh is grounded in subjective experience, so it would need an equivalent layer for that.
It seems like it'd be easier to use if you could see the intermediate steps, i.e. what the CLIP encoder "thought" the text prompt meant, and how it "thought" to turn that into an image prior. That could be done by decoding both into text again.
Also, if you could feed it some kind of "knowledge graph" of words and images rather than a sentence, you wouldn't be misunderstood the same way, but you'd presumably run into the limit of what its embeddings can contain.
I would argue that the risk-taking part of the creative process can also be indifferent to context. Highly impulsive processes need not be inclined to rationality.
Yes there needs to be an evaluator to guide and contextualize the results, but when most of the mechanical parts can be done so quickly (often most of the effort of the creative process), that's bound to lead to far higher output and willingness to get creative work done. You're right that it in itself is not strictly creative because it has no intentionality, but I've no doubt it will lead to more creativity.
As with typical art, time and energy are an investment. In this case the risk is taken by the person using it.
I guess 'creative' will always have intentionality built into it. Maybe an imagination unconstrained by purpose (e.g. dreaming) is not creative either.
> As with typical art, time and energy are an investment. In this case the risk is taken by the person using it.
This happens with or without creativity.
There are failure modes that showcase creativity, and there are failure modes that don't showcase creativity. These seem like they're largely the latter to me.
I'd prefer a real world stained glass robot though.
Even if it can't do the physical work (which can't be too hard), where a program told you what colors to buy, or you told it what you have, and it designed using that and printed/projected out a numbered tracing where to cut.
There's a Dall-E value add, turn it into a stained glass pattern and sell them.
When people talk about UBI and artists doing stuff... we have the tech now to tell a computer what we want and have a stained glass window come out of a factory. Maybe more expensive than humans today.
This is really awesome. I'm very excited to get access to DALL-E 2, is there any way to speed that up? Surprised someone hasn't used their access already to provide a "slow" community edition.
if the terms of use even remotely resemble those of the rest of openai, that probably violates them in about three different ways. it would be trivial for them to track down.
I’ve come to similar conclusions with midjourney. The most interesting outputs come from giving it very vague prompts and then doing a lot of manual curation of the results.
IIRC they provide some mechanism for fine-tuning: they showed a demo where you could erase portions, re-prompt and have it re-generate in the deleted area.
It seems like a pretty good general way to work toward specific requirements, though I haven't tried it.
Google claims to have a new DALL-E like model that does much better in precise instruction, due to using text models in different ways: https://imagen.research.google/
Unfortunately the impact of Imagen will be low like all the other Google models because, as keeps happening, they don't trust people enough to actually release it, not even in demo form:
"Imagen relies on text encoders trained on uncurated web-scale data, and thus inherits the social biases and limitations of large language models. As such, there is a risk that Imagen has encoded harmful stereotypes and representations, which guides our decision to not release Imagen for public use without further safeguards in place."
This seems to be emerging as a clear pattern - OpenAI trains an AI and then opens it to a group of select invitees (the usual suspects) with lots of opaque T&Cs that restrict what they're allowed to say about it [1]. For example, OpenAI forbids users from sharing any outputs that contain realistic faces even though the AI generates such pictures all the time, even when not requested [2]. This is reminiscent of how RDBMS vendors forbid users from publishing benchmarks and usually results in people giving OpenAI some stick for not living up to their name.
But OpenAI is still light years more open than Google, which routinely announces just months later that they've trained an AI far better than anything OpenAI produced, but doesn't provide any evidence beyond their own cherry picked examples. The justification given is always the same: the AI has learned un-woke things like the fact that certain jobs tend to be done by specific genders, and would generate examples based on that learnings. And because the Google researchers are crazy they think that seeing pictures of white male builders or young female nurses would be actually dangerous to the public, so they just don't provide any API access at all, not even to little select lists of friends.
IMO this is turning into a serious problem for AI research. It's already notoriously non-reproducible, but OpenAI's work is at least auditable. People can do experiments on DALL-E and see that it's real for themselves, explore the limits and come up with ways to handle them, as is happening here. They could in future build apps that use them. In contrast Google, who should by all rights be at the forefront of this space, keeps getting eclipsed by OpenAI again and again because they're now so woke that they've retreated into a tiny little purity bubble. They claim that they'll make their AIs available when they figured out how to brainwash them to have the right views, but they were claiming this is an "open problem" for years and never seem to release anything. So all the talk is about GPT-3 and DALL-E and Google's equivalents just get forgotten.
Is it non-reproducible though? Google did not release their model but they did publish their method, which they didn't have to do. People like lucidreams are already at work replicating them so everyone can benefit: https://github.com/lucidrains/DALLE2-pytorch
Yes, that's absolutely a good point worth repeating - Google is obviously under no obligation to publish anything or make any demos available. They can keep their research entirely proprietary if they want. There is maybe a question of whether this is what their researchers imagined, as many were brought to Google partly by the promise of being able to do their research in the open. But that's their problem.
The link you posted seems to reinforce my point though. It's a reimplementation of DALL-E, not anything done by Google. That seems to be how it goes, everyone tries to duplicate the OpenAI work even though Google's models are (or claim to be) more advanced. The mindshare value of making their demos available appears to be phenomenal.
Is it reproducible - well, only for flexible definitions of reproducible. If you try to follow the method in the paper and don't get results as good, is the issue the method or that you didn't follow it well enough? You can't follow it all that closely because they often aren't detailed enough and the training sets are constantly changing.
The issue here (for Google) is a bit different. If they were keeping their research proprietary because they wanted to turn it into a product and sell it, that's one thing and totally understandable. It's very expensive and needs some way to financially justify the cost. Their justification is quite different though and raises questions about their whole AI initiative. What's the point of creating SOTA AI trained on the internet if you're afraid of what people will use it for? Their efforts to make ideologically acceptable AI don't seem to have worked yet, nor has OpenAI been embarrassed by abuse. So OpenAI is powering ahead here and everyone talks about their models, whilst Google's languish in obscurity. Like, Scott Alexander is talking about the limits of DALL-E 2 that Google claim they already solved, but it's irrelevant because Scott can play with DALL-E and not Imagen.
I think I would definitely throw away most of the other stuff and restart with a theory that each of the people should have their heads on birds' bodies.
Tycho Brahe should definitely be an Owl.
Alexandra Elbakyan is evidently going to be a raven, however could probably ask her which bird she would want her head on.
which tells us for example that the Swan symbolizes "Light, twin flame, purity", so I guess the Swan would be lightness, but then who's head should be on it? I propose W.B Yeats because of the Wild Swans at Coole https://www.poetryfoundation.org/poems/43288/the-wild-swans-...
Not sure what birds the others should have their heads affixed to, but I think this is definitely the way I would go as soon as I saw Darwin's head on the finch.
The ones that made me laugh were the Tycho Brahe ones. It takes some time to realize that he's not looking through the telescope, but wistfully staring at it (or past it). Especially in the first moose picture, he seems to be looking at the telescope and asking himself "what the heck is this thing?!" - which is appropriate, since as other commenters mentioned, telescopes weren't yet invented during his lifetime...
"I think DALL-E assumes that anybody in the vicinity of a raven must be a Goth, and anybody in a library must be at least sort of Hermione Granger (I bet its corpus included a lot of Harry Potter fan art)"
I've spent 10 minutes trying to find a description of the training data for DALLE-2. The most I could find was that it was taken from 'publically-available' sources among others.
"nobody depicts a moose in stained glass. A man scrying the heavens through a telescope is exactly the sort of dignified thing people make stained glass windows about. A moose isn’t. So DALL-E loses confidence and decides you don’t really mean it should be stained glass style."
If the styles of the moose and the man were reversed, you could use the exact same explanation but end with "So DALL-E decides you don't really want the moose to be in stained glass style.".
When a human's head ends up on the body of the bird, such as with Darwin or Alexandra I have to imagine a game of telephone where a transcription error in a chain of tradition led to the emergence of an apocryphal figure and they rolled with it.
As an aside, how is the SSC community? Rewarding like HN? I accommodate irreverence well, but I don’t have a field guide for what this community might actually be about. Any tips from long time participants?
There can be a good number of opinionated armchair know it all‘s, but, like HN, if you’re willing to sift through comments there’s lots of great insight and references you might not come across on your own.
> I don’t have a field guide for what this community might actually be about.
It's part of "online rationalism" which is supposedly a community based around lesswrong where they practice thinking about things logically, with some techniques that work and some that don't.
More accurately it's a new religion that ascribes untrue magical powers to Bayes' theorem, probability theory, computers and AI research, gives people a life mission called "effective altruism", thinks if you program computers really well they'll become "superintelligences" and take over the world which needs to be stopped, and so on. This is why he says things about "rationality" in the post and is so approving about that Yudkowsky guy.
SSC years ago had some undeserved drama where the kind of online humanities-degree activist types who call you "tech bros" yelled at him, so I think the community also has people intentionally trying to make themselves into their image of a "tech bro" as a reaction to that.
(nb "religion" is not meant as an insult or a metaphor, starting new religions is what people do in Berkeley and this is just another one of them.)
>untrue magical powers to Bayes' theorem, probability theory, computers
Untrue magical powers? Just because people appreciate those things it doesn't mean they ascribe magical powers to them. That's like saying that HN ascribes magical powers to startups or chip architectures.
Well they're rationalists, so they ascribe it magical rationality powers.
They invoke Bayes' theorem for its property of being optimal, but it's optimal (and trivial) when you know the priors. The hard part is knowing those, and they don't, so they just use it in a metaphorical/mindset way that doesn't actually mean anything, pretend they don't have actual beliefs so they don't need to take responsibility for them, and say "prior" a lot. (https://metarationality.com/bayesianism-updating)
In general, using probability theory as a way of thinking doesn't work because it can't reject infinitesimally likely but infinitely bad futures, which is why they're overly concerned with imaginary future AIs torturing them, and it ignores "unknown unknowns" and can only handle enumerated futures you already thought of, which is why they have that odd tendency to start polycules because it seems logical and then their girlfriends unexpectedly leave them.
(That's also how you know it's a Berkeley new religion - weird sex practices.)
You are the only person ascribing it magical powers. The majority definitely do not and are well aware of the limitations, the problem with priors is talked about all the time in the community in much more detail.
>In general, using probability theory as a way of thinking doesn't work because it can't reject infinitesimally likely but infinitely bad futures
Another thing frequently discussed in those circles.
>(That's also how you know it's a Berkeley new religion - weird sex practices.)
Some people being sexually adventurous in California, the horror.
> Another thing frequently discussed in those circles.
That’s the problem, they shouldn’t be discussing it ;)
> Some people being sexually adventurous in California, the horror.
I could’ve said “joining cults like Leverage” instead of polycules but that would be mean. However like I did say, the problem with being a young man who gets into an open relationship because you rationally determined it was optimal is, you’re wrong and your girlfriend is going to leave you.
If you’re going to have weird sex, at least have a more fun reason for it. (There’s a tweet I want to link here about a rationalist nude sex party where they all sat around discussing immigration politics instead of having said adventures, but unfortunately I can’t remember how to spell Aeolla to find it.)
"you know the priors" - this is just fundamentally not how probabilistic reasoning works. You have some priors, which are of unclear accuracy, and then you iteratively update your beliefs on the basis of evidence and (probably) get, if not the right idea, then at least one that is less wrong. Hence the title!
If you had the right priors to start, you wouldn't need probability theory because you'd already know everything.
> it can't reject infinitesimally likely but infinitely bad futures
Sure it can, because that's value theory, not probability theory.
You don’t have priors or evidence - those are logical concepts that don’t exist in our fuzzy not-so-logical real world. You have some starting beliefs called priors, some new beliefs called evidence, and metaphorically applying one to the other causes you to be overly proud of the result because you feel like a rational process did it instead of your brain.
As the page I linked says, the real lesson is “don’t be too sure of stuff” but that doesn’t require pretending you’re doing math or joining any religions.
You seem to have some confused beliefs about what LessWrong people actually think. Theories like bayesian neurons aside, nobody thinks or advocates that people actually do bayesian updates as part of every thought. The point is that inasmuch as human thinking diverges from the formal math of probability, it will be less accurate - that there is such a thing as an idealized thought process, that thinking and learning has laws that can be understood and applied. That human minds aren't coincidental; that there are reasons why they work the way they do.
I've clocked some time online with the "rationalists". I have learned some interesting things.
But I was going to build a custom religion designed to make motte & bailey arguments in favor of it easy, it would probably look a lot like "rationalism". "It's not a religion, we just apply these precepts of rationality rationally! Look, the rational application of rational principles, rationally, is all but in the name!"
I actually agree that not everyone there treats it like a religion. There are people like me who just cruise by, pick some fruit, learn some things, apply some useful things, and get on with it. But there are absolutely also people for whom it is their religion.
Ironically, as IIRC some of the thought leaders have themselves pointed out (usually in the form of an agonized, hair-pulling sort of "Why, Bayes, why?" sort of post), there is very little rational reason to believe that improving rationality will lead to a lot of massively better life outcomes; the evidence of this is thin on the field. (Picking up some things around the edges, absolutely. But the hope that you suddenly Solved Life because you're "more rational" seems to be pretty poorly founded in reality.)
I agree, I just think that the way in which LessWrong goes wrong is largely an entirely different way in which the parent depicts us as going wrong. :P
> improving rationality will lead to a lot of massively better life outcomes
IMO, to defend this, I'd argue for most people there's maybe oom of ten decisions over a lifetime that in hindsight could turn out to be really important, and if you can keep a calm head and think about it in those moments, you can potentially get a lot of benefits. Now usually there'll be lots of advice for those, and the "rational" (in the advertised, not actual, sense) answer is not always the best one, but even just stepping back, thinking about it, trying to find out how to gather evidence about it, can already possibly help out a ton.
Most people get by fine doing well enough. And it doesn't take rationality to tell you not to invest your savings in scams, so the upside is limited. But for those few times where it really matters, there are some genuinely useful tools in there, IMO.
(Though personally, I'm in it for the philosophy.)
I agree with your description of (your?) beliefs, so if I implied anything else then I expressed myself poorly.
Unfortunately there isn't such a thing as an ideal thought process - that is, if you want to make a correct prediction about the world, there is no demonstratably correct method of doing that. This sort of thing has been tried before (logical positivism) and disproven (Wittgenstein and Gödel's theorem). It's notable that AI research is mostly getting places by just doing gigantic matrix multiplications instead of this.
It can be a useful exercise but it's not a good life philosophy - it's a bit robotic. And it gets cult-like if you add things on like "…and that's why you should live in our group home" or "…and that's why you should get rich and donate it all to evil AGI research".
This isn't very important to me though, it's just something I've noticed. It seem to come up a lot more lately because people reply to every ML story with "oh no the AIs are going to take our jobs" stuff, but still the largest impact it's had on the real world is Roko causing Elon Musk to date Grimes.
I think I meet more people who've developed life philosophies in reaction to knowing about LW ("post rats" or "meta rats") - mostly they explicitly decided to join religions instead of accidentally doing it - and for some reason all of them became Buddhists. That one surprises me.
> there is no demonstratably correct method of doing that
AIXI has some correctness proofs. In an admittedly esoteric sense, it does make "optimal use of information". I do think those parts philosophically hold up.
For an interesting view on Gödel, see Garrabrant's logical induction paper; you can bypass some aspects of that by working probabilistically.
> "…and that's why you should live in our group home"
People try things because they think they work, and that's why LW is a cult.
> "…and that's why you should get rich and donate it all to evil AGI research".
People recommend things because they think they're correct, and that's why LW is a cult.
Also, anti-evil AGI-research. That's kind of an important aspect.
I don't know what's going on with the postrats either.
Interesting that you know it as the SSC community, because Scott (TFA’s author)’s original site was Slate Star Codex. The new Substack where this was posted is Astral Codex Ten. Both are (near)anagrams for his nom de plume, Scott Alexander.
Anyway - I’ve lurked both for ~6-7 years. The best summary I can give is that Scott is an excellent and insightful author at his best, prone to certain idiosyncrasies, but who attracts a pretty thoughtful and generally pleasant commenter community.
The bias in astrange’s position is grounded in some factual truth, but I would argue that Scott’s commenter community is not entirely in the “rationality” camp, even if that’s a big part of how Scott reached his audience. The bigger part, imo, are the incredible series of articles he wrote circa 2014-2016 on a breadth of topics, many that still hold up.
I’d say check it out for the articles and see how you feel about the comments. Take astrange for what it’s worth, with a liberal dose of salt
You could send them to Fracture (https://fractureme.com) and while it wouldn't be stained glass, printing a picture of stained glass on actual glass is at least modern art.
I like this a lot, however, it would be great if he had created and actual robot that created the windows for real. My great grandfather was an expert at these and restored them for many churches; to see a robot learning and practicing this skill would be very cool.
The prompt was "A stained glass image of M.C. Escher working in his studio in non-Euclidean space", 600 iterations, 1024x768, and these are the first three ones it came up with. It's not perfect, but it definitely knows who he was!
Unfortunately some of these designs aren't feasible because the cuts won't work. It isn't possible to cut 'into' a piece of glass (imagine the 'mouth' of pacman) when making leadlights. You can see something like this in the first image where the flowers overlap the blue pane in the middle of the image. That blue pane is an example of an 'impossible' shape. At least this is according to an uncle of mine that has decades of professional experience designing, creating, and repairing stained glass and leadlights.
That's generally what is done I think. You do often see this where the shape is split in two. The disadvantage is that you end up with a line in a sometimes incongruous place, so a good designer would avoid it where possible.
As far as I can see that set of windows doesn't include any of the shapes I described. It does have 'cut outs' but they are curves, which don't have the same problem.
Edit: Actually, I see what you were referring to now. The two pieces at the top of each window in the door. I can't say that I know how that was accomplished. I've never seen something like that before. It might be revealing to see an image with a higher resolution.
Perhaps they broke out a start of concave curve and used a grinder to get it the rest of the way there*. It would probably be labor intensive and fragile but it might work.
The negativity over Dall-e's failures to make perfect images reminds me of the Louis CK stand up about the guy pissed about in-flight wifi not working, basically you're angry about not having this amazing technology you didn't know existed until this very moment.
I`m not sure that I agree or understand.
All things around me is a result of hundreds (thousands) years of development/culture/science/manufacturing etc.
I frequently reference Louis CK bit. It is so frequently applicable.
That said, I think it’s also a compliment to a tool when it does such a good job, that people criticise it for not doing a perfect job. The fact it’s actually doing the job at all is pretty wonderful. I thoroughly enjoyed seeing the results.
It depends a lot on what you think a good job actually looks like. This sort of prompt based regurgitation and remixing of data sets is super cool from a technical perspective and will maybe be useful one day but from someone mildly interested in this being an actual tool it needs to bridge that last 10%. That doesn't mean it has to be perfect at all. Which seems like a general summation of a lot of ML.
Negativity? The post is pretty optimistic and shows amazement of the general technology. Should people not talk about the failure modes and the places where it is subpar?
People adapt to getting an upgrade in life almost immediately. If you walked down the street and someone handed you $80 and the next guy $100, you'd be mad that he got more even though you just got free $$.
I'm trying to fix this aspect of myself. I was bowling in a tournament over the weekend that was way out of my league but I wanted the challenge. I was struggling all day and got kinda mad because I missed a really easy spare in my first good game of the day.
Lucky for me the guy on the other team was so positive and a much better bowler than me and he asked "That's ok, would you have been happy if I told you at the start of the game you'd have 107 in the 5th?" and I realized hell yeah I'd be happy. Same applies here. Of course I should be happy with a free $80 even if I know everyone else is getting $100.
I appreciated the discussion because most of the different posts I have read were about how amazing and life changing Dall-e was. This felt more like, "Wow, this is amazing, but it's not perfect."
I don’t think it’s accurate to say that there is only one image of Thomas Bayes in the training set. A quick image search reveals many duplicates and variations of this photo as well as some artistic interpretations of the image.
I don’t know how important that is, but it has a lot more examples of labelled data than just that one photo.
Very strange that this chap mentions Tychos. I've recently looked into an alternative cosmology (alternative to the Copernican model) based on tychos work:
I don't have time to read extensively, but an Earth-Sun binary with all the other planets orbiting the sun doesn't make sense given our understanding of gravity.
Also, NASA has flown shit to pretty much every planet. You think they could do that if the heliocentric model was wrong?
I find it interesting that it always seem to put the telescope near the mouth, presumably it has seen many more pictures of people drinking from a bottle than using a telescope.
The telescopes also look pretty weird.
145 comments
[ 0.25 ms ] story [ 255 ms ] threadAlso: if ML tech stalled now I expect we'd grow to have many Query Engineers, but if it doesn't do we still? For how long is this sort of experimentation what getting the best performance out of an AI will look like?
[1] https://en.wikipedia.org/wiki/Prompt_engineering
I don't see how anybody could reasonably hire someone for this position under this condition. How would one possibly discern between someone who can truly suss out what's going on and a techpriest who just recites learned dogma? I imagine it will definitely be an important skill, but I imagine that just like 'knowing how to google', it will be something that's just implicitly attached to a normal job.
Some people are better at it than others if you just watch the hashtags on Twitter. There will very clearly be value derived from it before we understand it. It’s very likely, in my opinion, that people will be hired for this type of work.
Like, since perfect synthesis is undetectable; it just sounds like what you're hearing is straightforward common sense. You only notice the process when it fails. So for sure, the book could have played with that a lot more, especially since Siri's tools are meant to be broken towards the end, when he's implied to be penning this. I think there's bits where the book suggests that's what's going on ("I can't tell you what it means, only what it says"), but it never came across well to me.
But then again, maybe that's just how a failure of synthesis would look. Like, you still want to understand what the book is saying, but if the mechanism is working it's invisible, and if it's not working, then you're still relying on it working to produce the thing you're reading. So it's just inherently hard to shine a light on the thing itself with itself, which of course plays into the central theme as well. Ie. because Siri is implied to not be fully conscious, it's hard for him to catch himself in the act of translating for us readers.
(Of course we may say that a book designed to be hard to read is still hard to read.)
https://en.wikipedia.org/wiki/Tycho_Brahe#Career:_observing_...
Edit: replacing 'Tycho Brahe' with 'Zombie Tycho Brahe' in the prompts could be one effective way to fix this
I think his nose was more or less skin-colored though, so the current pictures are okay. (DALL-E is supposedly programmed not to be able to generate real peoples' faces anyway.)
A woman who competed in a radio station’s contest to see how much water she could drink without going to the bathroom died of water intoxication, the coroner’s office said Saturday.
Should DALL-E play around with interpretations or not?
It seems to operate like a huge multidimensional map of an image space with some verbal tags stuck into it rather randomly, and with style transfer applied as an operator to merge different features in the results.
What it needs is an equivalent map for the concepts behind the signposts, and also a mapping between concepts and images.
The concept map would be very hard to train because a concept space that includes all of culture and history would be huge and messy.
A large concept map with a huge set of associations is one of the features of educated humans. But the concept mesh is grounded in subjective experience, so it would need an equivalent layer for that.
It's the CLIP text and image encoders of DALL-E 2.
> and also a mapping between concepts and images.
It's the "prior" module.
You guessed right but DALL-E 2 already has these functionalities in its design.
https://i0.wp.com/bdtechtalks.com/wp-content/uploads/2022/04...
Also, if you could feed it some kind of "knowledge graph" of words and images rather than a sentence, you wouldn't be misunderstood the same way, but you'd presumably run into the limit of what its embeddings can contain.
Yes there needs to be an evaluator to guide and contextualize the results, but when most of the mechanical parts can be done so quickly (often most of the effort of the creative process), that's bound to lead to far higher output and willingness to get creative work done. You're right that it in itself is not strictly creative because it has no intentionality, but I've no doubt it will lead to more creativity.
There's some connection but they're largely separate.
Whether something is on a good path or not is part of the curation process.
There's a lot of 'creativity' to showcase, but that doesn't seem to be what's causing these failures.
I guess 'creative' will always have intentionality built into it. Maybe an imagination unconstrained by purpose (e.g. dreaming) is not creative either.
This happens with or without creativity.
There are failure modes that showcase creativity, and there are failure modes that don't showcase creativity. These seem like they're largely the latter to me.
I'd prefer a real world stained glass robot though.
Even if it can't do the physical work (which can't be too hard), where a program told you what colors to buy, or you told it what you have, and it designed using that and printed/projected out a numbered tracing where to cut.
There's a Dall-E value add, turn it into a stained glass pattern and sell them.
When people talk about UBI and artists doing stuff... we have the tech now to tell a computer what we want and have a stained glass window come out of a factory. Maybe more expensive than humans today.
I dunno, I think a stained glass window of a finch with Charles Darwin's head is pretty great, personally
The same can be said with respect to how a great many treat most of their relationships, both personally, professionally, politically...
It seems like a pretty good general way to work toward specific requirements, though I haven't tried it.
Unfortunately the impact of Imagen will be low like all the other Google models because, as keeps happening, they don't trust people enough to actually release it, not even in demo form:
"Imagen relies on text encoders trained on uncurated web-scale data, and thus inherits the social biases and limitations of large language models. As such, there is a risk that Imagen has encoded harmful stereotypes and representations, which guides our decision to not release Imagen for public use without further safeguards in place."
This seems to be emerging as a clear pattern - OpenAI trains an AI and then opens it to a group of select invitees (the usual suspects) with lots of opaque T&Cs that restrict what they're allowed to say about it [1]. For example, OpenAI forbids users from sharing any outputs that contain realistic faces even though the AI generates such pictures all the time, even when not requested [2]. This is reminiscent of how RDBMS vendors forbid users from publishing benchmarks and usually results in people giving OpenAI some stick for not living up to their name.
But OpenAI is still light years more open than Google, which routinely announces just months later that they've trained an AI far better than anything OpenAI produced, but doesn't provide any evidence beyond their own cherry picked examples. The justification given is always the same: the AI has learned un-woke things like the fact that certain jobs tend to be done by specific genders, and would generate examples based on that learnings. And because the Google researchers are crazy they think that seeing pictures of white male builders or young female nurses would be actually dangerous to the public, so they just don't provide any API access at all, not even to little select lists of friends.
IMO this is turning into a serious problem for AI research. It's already notoriously non-reproducible, but OpenAI's work is at least auditable. People can do experiments on DALL-E and see that it's real for themselves, explore the limits and come up with ways to handle them, as is happening here. They could in future build apps that use them. In contrast Google, who should by all rights be at the forefront of this space, keeps getting eclipsed by OpenAI again and again because they're now so woke that they've retreated into a tiny little purity bubble. They claim that they'll make their AIs available when they figured out how to brainwash them to have the right views, but they were claiming this is an "open problem" for years and never seem to release anything. So all the talk is about GPT-3 and DALL-E and Google's equivalents just get forgotten.
[1] https://www.lesswrong.com/posts/uKp6tBFStnsvrot5t/what-dall-...
[2] https://www.lesswrong.com/posts/uKp6tBFStnsvrot5t/what-dall-...
The link you posted seems to reinforce my point though. It's a reimplementation of DALL-E, not anything done by Google. That seems to be how it goes, everyone tries to duplicate the OpenAI work even though Google's models are (or claim to be) more advanced. The mindshare value of making their demos available appears to be phenomenal.
Is it reproducible - well, only for flexible definitions of reproducible. If you try to follow the method in the paper and don't get results as good, is the issue the method or that you didn't follow it well enough? You can't follow it all that closely because they often aren't detailed enough and the training sets are constantly changing.
The issue here (for Google) is a bit different. If they were keeping their research proprietary because they wanted to turn it into a product and sell it, that's one thing and totally understandable. It's very expensive and needs some way to financially justify the cost. Their justification is quite different though and raises questions about their whole AI initiative. What's the point of creating SOTA AI trained on the internet if you're afraid of what people will use it for? Their efforts to make ideologically acceptable AI don't seem to have worked yet, nor has OpenAI been embarrassed by abuse. So OpenAI is powering ahead here and everyone talks about their models, whilst Google's languish in obscurity. Like, Scott Alexander is talking about the limits of DALL-E 2 that Google claim they already solved, but it's irrelevant because Scott can play with DALL-E and not Imagen.
Tycho Brahe should definitely be an Owl.
Alexandra Elbakyan is evidently going to be a raven, however could probably ask her which bird she would want her head on.
Since this is virtues we can also go looking for help in the symbolism of birds https://worldbirds.com/bird-symbolism/
which tells us for example that the Swan symbolizes "Light, twin flame, purity", so I guess the Swan would be lightness, but then who's head should be on it? I propose W.B Yeats because of the Wild Swans at Coole https://www.poetryfoundation.org/poems/43288/the-wild-swans-...
Not sure what birds the others should have their heads affixed to, but I think this is definitely the way I would go as soon as I saw Darwin's head on the finch.
You get what you pay for.
"nobody depicts a moose in stained glass. A man scrying the heavens through a telescope is exactly the sort of dignified thing people make stained glass windows about. A moose isn’t. So DALL-E loses confidence and decides you don’t really mean it should be stained glass style."
If the styles of the moose and the man were reversed, you could use the exact same explanation but end with "So DALL-E decides you don't really want the moose to be in stained glass style.".
So there is no need for pluralised word.
And we are back to moose!
It's part of "online rationalism" which is supposedly a community based around lesswrong where they practice thinking about things logically, with some techniques that work and some that don't.
More accurately it's a new religion that ascribes untrue magical powers to Bayes' theorem, probability theory, computers and AI research, gives people a life mission called "effective altruism", thinks if you program computers really well they'll become "superintelligences" and take over the world which needs to be stopped, and so on. This is why he says things about "rationality" in the post and is so approving about that Yudkowsky guy.
SSC years ago had some undeserved drama where the kind of online humanities-degree activist types who call you "tech bros" yelled at him, so I think the community also has people intentionally trying to make themselves into their image of a "tech bro" as a reaction to that.
(nb "religion" is not meant as an insult or a metaphor, starting new religions is what people do in Berkeley and this is just another one of them.)
Untrue magical powers? Just because people appreciate those things it doesn't mean they ascribe magical powers to them. That's like saying that HN ascribes magical powers to startups or chip architectures.
They invoke Bayes' theorem for its property of being optimal, but it's optimal (and trivial) when you know the priors. The hard part is knowing those, and they don't, so they just use it in a metaphorical/mindset way that doesn't actually mean anything, pretend they don't have actual beliefs so they don't need to take responsibility for them, and say "prior" a lot. (https://metarationality.com/bayesianism-updating)
In general, using probability theory as a way of thinking doesn't work because it can't reject infinitesimally likely but infinitely bad futures, which is why they're overly concerned with imaginary future AIs torturing them, and it ignores "unknown unknowns" and can only handle enumerated futures you already thought of, which is why they have that odd tendency to start polycules because it seems logical and then their girlfriends unexpectedly leave them.
(That's also how you know it's a Berkeley new religion - weird sex practices.)
>In general, using probability theory as a way of thinking doesn't work because it can't reject infinitesimally likely but infinitely bad futures
Another thing frequently discussed in those circles.
>(That's also how you know it's a Berkeley new religion - weird sex practices.)
Some people being sexually adventurous in California, the horror.
That’s the problem, they shouldn’t be discussing it ;)
> Some people being sexually adventurous in California, the horror.
I could’ve said “joining cults like Leverage” instead of polycules but that would be mean. However like I did say, the problem with being a young man who gets into an open relationship because you rationally determined it was optimal is, you’re wrong and your girlfriend is going to leave you.
If you’re going to have weird sex, at least have a more fun reason for it. (There’s a tweet I want to link here about a rationalist nude sex party where they all sat around discussing immigration politics instead of having said adventures, but unfortunately I can’t remember how to spell Aeolla to find it.)
If you had the right priors to start, you wouldn't need probability theory because you'd already know everything.
> it can't reject infinitesimally likely but infinitely bad futures
Sure it can, because that's value theory, not probability theory.
As the page I linked says, the real lesson is “don’t be too sure of stuff” but that doesn’t require pretending you’re doing math or joining any religions.
But I was going to build a custom religion designed to make motte & bailey arguments in favor of it easy, it would probably look a lot like "rationalism". "It's not a religion, we just apply these precepts of rationality rationally! Look, the rational application of rational principles, rationally, is all but in the name!"
I actually agree that not everyone there treats it like a religion. There are people like me who just cruise by, pick some fruit, learn some things, apply some useful things, and get on with it. But there are absolutely also people for whom it is their religion.
Ironically, as IIRC some of the thought leaders have themselves pointed out (usually in the form of an agonized, hair-pulling sort of "Why, Bayes, why?" sort of post), there is very little rational reason to believe that improving rationality will lead to a lot of massively better life outcomes; the evidence of this is thin on the field. (Picking up some things around the edges, absolutely. But the hope that you suddenly Solved Life because you're "more rational" seems to be pretty poorly founded in reality.)
> improving rationality will lead to a lot of massively better life outcomes
IMO, to defend this, I'd argue for most people there's maybe oom of ten decisions over a lifetime that in hindsight could turn out to be really important, and if you can keep a calm head and think about it in those moments, you can potentially get a lot of benefits. Now usually there'll be lots of advice for those, and the "rational" (in the advertised, not actual, sense) answer is not always the best one, but even just stepping back, thinking about it, trying to find out how to gather evidence about it, can already possibly help out a ton.
Most people get by fine doing well enough. And it doesn't take rationality to tell you not to invest your savings in scams, so the upside is limited. But for those few times where it really matters, there are some genuinely useful tools in there, IMO.
(Though personally, I'm in it for the philosophy.)
Unfortunately there isn't such a thing as an ideal thought process - that is, if you want to make a correct prediction about the world, there is no demonstratably correct method of doing that. This sort of thing has been tried before (logical positivism) and disproven (Wittgenstein and Gödel's theorem). It's notable that AI research is mostly getting places by just doing gigantic matrix multiplications instead of this.
It can be a useful exercise but it's not a good life philosophy - it's a bit robotic. And it gets cult-like if you add things on like "…and that's why you should live in our group home" or "…and that's why you should get rich and donate it all to evil AGI research".
This isn't very important to me though, it's just something I've noticed. It seem to come up a lot more lately because people reply to every ML story with "oh no the AIs are going to take our jobs" stuff, but still the largest impact it's had on the real world is Roko causing Elon Musk to date Grimes.
I think I meet more people who've developed life philosophies in reaction to knowing about LW ("post rats" or "meta rats") - mostly they explicitly decided to join religions instead of accidentally doing it - and for some reason all of them became Buddhists. That one surprises me.
AIXI has some correctness proofs. In an admittedly esoteric sense, it does make "optimal use of information". I do think those parts philosophically hold up.
For an interesting view on Gödel, see Garrabrant's logical induction paper; you can bypass some aspects of that by working probabilistically.
> "…and that's why you should live in our group home"
People try things because they think they work, and that's why LW is a cult.
> "…and that's why you should get rich and donate it all to evil AGI research".
People recommend things because they think they're correct, and that's why LW is a cult.
Also, anti-evil AGI-research. That's kind of an important aspect.
I don't know what's going on with the postrats either.
Anyway - I’ve lurked both for ~6-7 years. The best summary I can give is that Scott is an excellent and insightful author at his best, prone to certain idiosyncrasies, but who attracts a pretty thoughtful and generally pleasant commenter community.
The bias in astrange’s position is grounded in some factual truth, but I would argue that Scott’s commenter community is not entirely in the “rationality” camp, even if that’s a big part of how Scott reached his audience. The bigger part, imo, are the incredible series of articles he wrote circa 2014-2016 on a breadth of topics, many that still hold up.
I’d say check it out for the articles and see how you feel about the comments. Take astrange for what it’s worth, with a liberal dose of salt
* https://jossi.avkrok.net/Escher_0.png
* https://jossi.avkrok.net/Escher_1.png
* https://jossi.avkrok.net/Escher_2.png
The prompt was "A stained glass image of M.C. Escher working in his studio in non-Euclidean space", 600 iterations, 1024x768, and these are the first three ones it came up with. It's not perfect, but it definitely knows who he was!
[1] https://github.com/nerdyrodent/VQGAN-CLIP [2] https://github.com/CompVis/latent-diffusion [3] https://imgur.com/a/DjQYLUz
I.e. no deepfakes…
[0] https://theglassroom.co.nz/wp-content/uploads/2019/04/23f059...
Edit: Actually, I see what you were referring to now. The two pieces at the top of each window in the door. I can't say that I know how that was accomplished. I've never seen something like that before. It might be revealing to see an image with a higher resolution.
I think this comment under The Charles Darwin Finch is one of the funniest things I've read this year.
That said, I think it’s also a compliment to a tool when it does such a good job, that people criticise it for not doing a perfect job. The fact it’s actually doing the job at all is pretty wonderful. I thoroughly enjoyed seeing the results.
It takes 2 weeks to adapt to a loss though.
Humans are strange.
Lucky for me the guy on the other team was so positive and a much better bowler than me and he asked "That's ok, would you have been happy if I told you at the start of the game you'd have 107 in the 5th?" and I realized hell yeah I'd be happy. Same applies here. Of course I should be happy with a free $80 even if I know everyone else is getting $100.
https://www.tychos.space/
I'm unfamiliar with this area, but it seems impressive to me that a lot of astronomical quirks are resolved:
https://book.tychos.space/chapters/31-list-of-puzzles
Also, NASA has flown shit to pretty much every planet. You think they could do that if the heliocentric model was wrong?