I thought the same thing but I think the commenter is making a joke, but I could be wrong.
I think they are suggesting that things like this (neural nets etc) work using bias, and by removing "bias" the developers are making the product worse.
Unfortunately, the method OpenAI may be using to reduce bias (by adding words to the prompt unknown to the user) is a naive approach that can affect images unexpectedly and outside of the domain OpenAI intended: https://twitter.com/rzhang88/status/1549472829304741888
I have also seeing some cases where the bias correction may not be working at all, so who knows. And it's why transparancy is important.
What a fascinating hack. I mean, yeah, naive and simplistic and doesn't really do anything interesting with the model itself, but props to the person who was given the "make this more diverse" instruction and said "okay, what's the simplest thing that could possibly work? What if I just append some races and genders onto the end of the query string, would that mostly work?" and then it did! Was it a GOOD idea? Maybe not. But I appreciate the optimization.
"Starting today, users get full usage rights to commercialize the images they create with DALL·E, including the right to reprint, sell, and merchandise. This includes images they generated during the research preview."
I assumed this was going to be the sticking point for wider usage for a long time. They're now saying that you have full rights to sell Dall-E 2 creations?
Previously, OpenAI asserted they owned the generated images, so the new licensing is a shift in that aspect. GPT-3 also has a "you own the content" clause as well.
Of course, that clause won't deter a third party from filing a lawsuit against you if you commercialize a generated image too close to something realistic, as the copyrights of AI generated content still hasn't been legally tested.
AFAIK only people can own copyright (the monkey selfie case tested this), and machine-generated outputs don't count as creative work (you can't write an algorithm that generates every permutation of notes and claim you own every song[1]), so DALL-E-generated images are most likely copyright-free. I presume OpenAI only relies on terms of service to dictate what users are allowed to do, but they can't own the images, and neither can their users.
The monkey selfie was not derived from millions of existing works, and that is the difference. If an artist has a well-known art style, and this algorithm was trained on it and can copy that style, would the artist have grounds to sue? I don't know.
Right but if that work isn’t significantly changed from the source, it could be ruled as infringement. DALL-E cannot tell the users (afaik) if a result is close to any source material.
> If an artist has a well-known art style, and this algorithm was trained on it and can copy that style, would the artist have grounds to sue? I don't know.
While nothing has been commercialized yet on the DALLE2 subreddit, I know that it can do Dave Choe's work remarkably well. I also saw Alex Gray's work to be close, but not really identical either. It wasn't as intricate as his work is.
It will be interesting if this takes off and you have a sort of Banksy effect take over where unless it's a physical piece of art it doesn't have much value and is only made all the better because of some sort polemic attached to it, eg Girl with balloon.
I'm going to guess there's not going to be much value placed on anything out of DALLE for a long while. Digital art is typically worth much less than physical art and I would say these GAN images are going to worth less than digital art generated by human hand.
There will be outliers of course but I would be shocked if there's much of a market in it for at least the present.
I think the value will be in work produced that gets attached to things which are being sold. So, a book cover or an album cover. If a best selling novel used artwork from this system and it happened to be a very close copy of existing work, I could imagine the author of the original work suing for royalties.
Sure but if you just release a basic copy of a Taylor Swift song you will get sued to oblivion. So the law seems (IANAL) to care about how similar your work is to existing works. DALL-E does not seem capable of showing you the work that influenced a result, so users will have no idea if a result might be infringing. What this means to me is that with many users, some of the results would be legally infringing.
Even if you imitate someone's style intentionally, they don't have grounds to sue. Style isn't copyrightable in the US. Whether DALL-E outputs are a derivative work is a different question, though
If this were a concern, a user can easily bypass this by having a work-for-hire person add a minor transform layer on top of the DALL-E generated images right?
> DALL-E-generated images are most likely copyright-free
The US Copyright Office did make a ruling that might suggest that recently[1], but crucially, in that case, the AI "didn't include an element of human authorship." The board might rule differently about DALL-E because the prompts do provide an opportunity for human creativity.
And there's another important caveat that the felixreda.eu link seems to miss. DALL-E output, whether or not it's protected by copyright, can certainly infringe other copyrights, just like the output of any other mechanical process. In short, Disney can still sue if you distribute DALL-E generated images of Marvel characters.
DALL-E can generate recognizable pictures of Homer Simpson, Batman and other commercial properties. Such images could easily be considered derivative works of the original copyrighted images that were used as training input. I'm sure there are plenty of corporate IP lawyers ready to argue the point at court.
I'm kind of surprised that no one had found "verbatim copy" cases as were made with GitHub Copilot. Such exact copies in photography are likely easier to go for than with code snippets.
It might be interesting to find an image in the training set with a long, very unique description, and try that exact same description as input in DALL·E 2.
Of course it's unlikely to produce the exact same image, or if it does, you've also discovered an incredible image compression algorithm.
Image generating artificial intelligence is very analogous to a camera.
Both technologies have billions of dollars of R&D and tens of thousands of engineers behind supply chains necessary to create the button that a user has the press.
I think they are reacting to competition. MidJourney is amazing, was easier to get into, gives you commercial rights, and frankly I found more fun to use and even better output in most instances.
MidJourney definitely struggles more with complex prompts from what I saw. If you like the output more, that’s subjective, but I think DALL•E is the leader in the space by a wide margin.
I think both have strengths and weaknesses, but I don’t disagree DALL-E in most instances is technically better at matching prompts. But I often enjoyed, artistically, the results of MidJourney more; it just felt fun to use and explore.
The only thing I don’t like about MidJourney is the Discord based interface. I think I can grok why Dave chose this route as it bakes in an active community element and allows users to pick up prompt engineering techniques osmotically… but I’d prefer a clean DALL-E style app and cli / api access.
In case you don’t know, you can at least PM the MidJourney bot so you have an uncluttered workspace.
It’s clearly personally preference, but I loathe Discord but love it for MidJourney. As you said, there’s an interactive element where I see other people doing cool things and adapting part of their prompts and vice versa. It really is fun. And when you do it in a PM, you have all your efforts saved. DALL-E is pretty clunky in that you have to manually save an image or lose it once your history rolls off.
Thanks. Yeah fair point; I haven’t ponied up for a subscription yet so am still stuck in public channels and often find my generations get lost in the stream. Imagine you’re right and having the PM option would change the experience drastically for the better albeit still within Discord’s visually chaotic environment.
I've completely changed my mind after spending the last few days neck deep in it around the clock. Sleep is overrated! MidJourney is awesome and the way it's implemented within Discord is a masterstroke of elegant simplicity.
I have access to both and they're good for different things. DALL-E seems somewhat more likely to know what you mean. Midjourney seems better for making interesting fantasy and science fiction environments.
For comparison, I tried generating images of accordions. Midjourney doesn't really understand that an accordion has a bellows [1]. DALL-E manages to get the right shape much of the time, if you don't look too closely: [2], [3]. Neither of them knows the difference between piano and button accordions.
Neither of them can draw a piano keyboard accurately, but DALL-E is closer if you don't look too hard. (The black notes aren't in alternating groups of two and three.)
Neither of them understands text; text on a sign will be garbled. Google's Parti project can do this [4], but it's not available to the public.
I expect DALL-E will have many people sign up for occasional usage, because if you don't use it for a few months, the free credits will build up. But Midjourney's pricing seems better if you use it every day?
I guess it depends on what you like/enjoy? It's not good at photorealistic, but it comes up with some pretty entertaining (and pretty?) 'arty' type stuff. I go on regularly just to play around for fun.
It also means there will possibly be another renaissance of fully automated, mass generated NFTs and tons of derivatives and remixes flooding the NFT market in an attempt to pump the NFT hype again.
It doesn't matter, OpenAI wins anyway as these companies will pour hundreds of thousands into generated images.
It seems that the NFT grift is about to be rebooted again, such that it isn't going to die that quickly. But still, eventually 90% of these JPEG NFTs will die anyway.
These high photorealistic images can be generated on a mass-scale, completely automated without a human which ultimately cuts the need for an artist to do that.
They will be replaced by DALL·E 2 for creating these illustrations, book covers, NFT variants, etc opening up the whole arena to anyone to do this themselves. All it takes is to describe what they want in text and less than a minute, the work is delivered as little as $15.
OpenAI still wins either way. If a crypto company goes to using DALL·E 2 to generate photorealistic NFTs, they won't stop them and they will take the money.
> "Starting today, users get full usage rights to commercialize the images they create with DALL·E, including the right to reprint, sell, and merchandise. This includes images they generated during the research preview."
>> And I just used it to create cover art for a book published in Amazon :)
Man... what a missed opportunity for Altman... he could have had a really good cryptocurrency/token with a healthy ecosystem and a creative based community if he didn't push this Worldcoin biometric harvesting BS had he just waited for this to release and coupled it with access to GPT.
This is the kind of thing that Web3 (a joke) was pushing for all along: revolutionary tech that the everyday person can understand with it's own token based ecosystem for access with full creative rights from the prompts.
I wonder if he stepped down from Open AI and put it in a figurehead as CEO could this still work?
> Why is using a token better than using money, in this case?
It would be better for OpenAI if it can monetize not just its subscription based model via a token to pay for overhead and for further R/D but also for it's ability to issue tokens it can freely exchange for utility on it's platform for exclusive access outside of it's capped $15 model and allow for pay as you go models for those who don't have access to it like myself as it's limited to 1 million users.
I don't want an account, and I think that type of gatekeeping wasn't cool during the gmail days either and I had early access back then too, but I'd still personally buy $100s of dollars worth of prompts right now since I think it is fascinating use of NLP and I'm just one of many missed opportunities and represent a lost userbase who just want access for specific projects. By doing this they can still retain the caps of useage on their platform and expand and contract them as they see fit without excluding others.
This in turn could justify the continual investment from the VC World into these projects (under the guise of web3) and allow them to scale into viable businesses and further expand the use of AI/ML into other creative spaces, which as a person studying AI and ML and a background in BTC, is what we all wanted to see instead of these aimless bubbles in things like Solana or yield farming via fake DeFi projects like Calesius that we've seen.
It would legitimize the use of a token for use of an ecosystem model outside of BTC, which to be honest doesn't really exist and has still a tarnished view with all these failed projects, while gaining reception amongst a greater audience since it's captivated so many since it's release.
$0.13/prompt can only be useful for artists/end users. Anyone thinking about using this at scale would need a 20/30x reduction in price. But there's still no API available so I think that will change with time. Maybe they will add different tiers based on volume.
That is a fair point. I don't think the pricing is unreasonable, but it feels limiting. You could try 1000 variations until you find what you need perfectly, but in that pricing model users will be induced to use less the tool, not more.
I'd prefer an option to pay like 200 usd/year to use unlimited. And maybe have a price per use only in the API.
edit: this pricing model also makes it expensive to learn to use the tool.
Until you consider the level of demand for this product, which is surely higher than OpenAI can scale to with the number of GPUs they have. If they price it lower they’ll be overwhelmed.
I wonder if they have plans to allow SVG exports in the future. I mean, the file size would probably be ridiculous in a lot of the cases, but for my use case I wouldn't mind it. And sucks about the watermark, maybe they will introduce an option to pay for removing it.
SVG exports would only be meaningful if the model is generating vector images, which are then converted to bitmaps. I highly doubt that's the case, but perhaps someone who has actually looked at the model structure can confirm?
SVG isn't really possible with the model architecture they're using. The diffusion+upscaling step basically outputs 1024x1024 pixels; at no point does the model have a vector representation.
I suppose it's possible that at some point they'll try to make an image -> svg translation model?
One of the commercial use cases this post mentions is authors who want to add illustrations to children's stories.
I wonder if there is a way for DALL-E to generate a character, then persist that character over subsequent runs. Otherwise, it would be pretty difficult to generate illustrations that depict a coherent story.
Example ...
Image 1 prompt: A character named Boop, a green alien with three arms, climbs out of its spaceship.
Image 2 prompt: Boop meets a group of three children and shakes hands with each one.
You mean just generate a single large image with all the stuff you want for the whole story, and then use cropping and inpainting to get only the piece you want for each page?
You can't do that. I can't see this working well for children's book illustrations unless the story was specifically tailored in a way that makes continuity of style and characters irrelevant.
As an aside, Ursula Vernon did pretty well under the constraint you described. She set a comic in a dreamscape and used AI to generate most of the background imagery: https://twitter.com/UrsulaV/status/1467652391059214337
It's not the "specify the character positions in text" proposed, but still a neat take on using this sort of AI for art.
That's disappointing given up until this point you could have 50 free uses per 24h. I expected it to get monetized eventually, but not so fast and drastically. Well, still had my fun and have to say the creations are so good it's often mind blowing there's an AI behind it.
Honestly, it is probably just that expensive to run. You can’t expect someone to hand you free compute of significant value and directly charging for it is a lot better than other things they could do.
Not correct. They have a for-profit entity now. That's why there is a huge incentive to monetize. Any for-profit investment gain is capped at 100x, with the rest required to go to their nonprofit.
This commercialization is just as I predicted in my substack post 2 days ago that hit the front page of Hacker News: https://aifuture.substack.com/p/the-ai-battle-rages-on
> Starting today, users get full usage rights to commercialize the images they create with DALL·E, including the right to reprint, sell, and merchandise. This includes images they generated during the research preview.
So DALL·E 2 is going to restart, revive and cause another renaissance of fully automated and mass generated NFTs, full of derivatives and remixing etc to pump up the crypto NFT hype squad?
Either way, OpenAI wins again as these crypto companies are going to pour tens of thousands of generated images to pump their NFT griftopia off of life support, reconfirming that it isn't going to die that easily.
Regardless of this possible revival attempt, 90% of these JPEG NFTs will eventually still die.
I don't see why there's any credible reason to expect that DALL-E will do anything at all to help those promoting the NFT silliness. Two separate issues.
> I think it has not been trained on NFT art (crypto punks and so on).
How exactly are you defining NFT art?
I mean, it can literately be anything: Dorsey sold a screencap of his 1st tweet, Nadya from Pussy Riot did some creative stuff, and the Ape crap was the bulk of this stuff that got passed around.
I think what can be gleaned from that short-lived non-sense is that value is subjective and that the quality of a valuabe piece of 'art' is equally as hard to define. Much the same with its predecessor: cryptokitties.
Heads up: I think you meant "in vain" rather than "in vail". However, a similar phrase is "to no avail" which also means that something was not successful.
Something I haven’t seen anyone talking about with these huge models: how do future models get trained when more content online is model generated to start with? Presumably you don’t wanna train a model on autogenerated images or text, but you can’t necessarily know which is which.
In this situation, the low-background steel is the MS-COCO dataset, associated with the Fréchet inception distance computed by comparing the statistical divergence between the high-level vector outputs of passing MS-COCO images through Google’s InceptionV3 classifier, and passing DALL-E images (or its competitors) through it.
For now at least, there is a detectable difference in variety.
But how would you know? A random string of text or an image with the watermark removed is going to be very hard to distinguish generated from human written.
At least cleaning it up is an embarrassingly parallel problem, so if you had the resources to throw incentives at millions of casual gamers, you might make a nice dent on Clip.
Alternatively, making a captcha where half the data is unlabeled, and half is labeled, forcing users to categorize data for you as they log into accounts.
> Getting humans to refine your data is the best solution right now
Source ?
All those big models are trained with data for which the source is not known or vetted. The amount of data needed is not human-refinable.
For example for language models we train mostly on subsets of CommonCrawl + other things. CommonCrawl data is “cleaned” by filtering out known bad sources and with some heuristics such as ratio of text to other content, length of sentences etc.
The final result is a not too dirty but not clean huge pile of data that comes from millions of sources that no human as vetted and that no one in the team using the data knows about.
The same applies to large images dataset, e.g. Laon 400m that also comes from CommonCrawl and is not curated.
This precise thing is causing a funny problem in specialty areas. People are using e.g. Google Lens to identify plants, birds and insects, which sometimes returns wrong answers e.g. say it sees a picture of a Summer Tanager and calls it a Cardinal. If the people then post "Saw this Cardinal" and the model picks up that picture/post and incorporates it into its training set, it's just reinforcing the wrong identification..
That's not really a new problem, though. At one point someone got some bad training data about an old Incan town, the misidentification spread, and nowadays we train new human models to call it Macchu Picchu.
It's a cybernetic feedback system. Dalle is used to create new images, the images that people find most interesting and noteworthy get shared online, and reincorporated into the training data, but now filtered through human desire.
I wonder if human artists can demand that their work not be used for modelling. So as the robots are stuck using older styles for their creations, the humans will keep creating new styles of art.
One interesting comment about this is that some models actually benefit from being fed their own output. Alphafold for instance was fed with its own 'high likelihood' outputs (as demis hassabis described in his lex friedman interview).
Training on auto generated images collected off the Internet is gonna be fine for a while since the images surfacing will be curated (ie. selected as good/interesting/valuable) still mostly by humans.
I think with the terms requiring explicitly telling which images/parts were generated, they could be filtered out and prevent a feedback loop of "generated in/generated out" images. I'm sure there will be some illegal/against terms of use cases there but the majority should represent fair use.
Have you tried any of the "human or Dall-E" tests?
How did you score?
I only scored as well as I did because I knew the kind of stylistic choices to look out for. In terms of "quality" I really don't understand how you've reached this conclusion.
> Reducing bias: We implemented a new technique so that DALL·E generates images of people that more accurately reflect the diversity of the world’s population. This technique is applied at the system level when DALL·E is given a prompt about an individual that does not specify race or gender, like “CEO.”
Will it do it "more accurately" as they claim? As in, if 90% of CEOs are male, then the odds of a CEO being male in a picture is 90%? Or less "accurately reflect the diversity of the world’s population" and show what they would like the real world to be like?
The latter. Here's what we, a small number of people, think the world should look like according to our own biases and information bubble in the current moment. We will impose our biases upon you, the unenlightened masses who must be manipulated for your own good. And for god sakes, don't look for photos of the US Math team or NBA Basketball or compare soccer teams across different countries and cultures.
You are correct but that's not what anyone's discussing.
If I search for "food", the reasonable result would be to get images that represent food according to its actual proportions of real life. E.g. if Pizza is the most common food at 10% prevalence, 10% of the images should be pizza.
That's not what OpenAI are doing.
They are introducing crafted biases to create images that deliberately misrepresent what the world looks like, and instead represent what they believe the world ought to look like.
--
You also need some reason why diversity "of this" is important but not diversity "of that". Why is diversity of race and sex so critical, but not diversity of age, height, disability? Should a search for "basketball player" yield 1/2 able-bodied people and 1/2 wheelchair basketball players? Why?
Then try to answer where you came up with the categories you do want depicted. Why are the races what they are? Should "basketball player" include half whites and half black people? Or maybe split in 3, white/black/Asian? Why not Australian Aborigines, native Americans, or Persians - so we can divide into 6? If you don't add Indian people to your list then, is that racist against them? How did you decide what must be represented, in what proportions, and what's okay to leave out?
As competitors with lower price points prop up, you'll see everyone ditch models with "anti bias" measures and take their $ somewhere else. Or maybe we'll get some real solution, that adds noise to the embeddings, and not some half assed workaround to the arbitrary rules that your resident AI Ethicist comes up with.
Add after. So you can see the added words by making a prompt like "a person holding a sign saying ", and then the sign says the extra words if they are added.
Would only work for positive biases where if they actually want to equalize it then it needs to be adding the opposite to negative biases.
To counteract the bias of their dataset they need to have someone sitting there actively thinking in bias to counteract the bias with anti-bias seasoning for every bias causing term. Feel bad for whatever person is tasked with that job.
Could always just fix your dataset, but who's got time and money to do that /s
Yes the quality of surrealist generations went down with that change suddenly including gender and race into prompts that I really didn't want anything specific in. Like a snail radio DJ, and suddenly the microphone is a woman of colours head.. I understand the intention but I want this to be a default on but you can turn it off thing.
It's also odd since you'd think that this would be an issue solved by training with representative images in the first place.
If you used good input you'd expect an appropriate output, I don't know why manual intervention would be necessary unless it's for other purposes than stated. I suspect this is another case where "diversity" simply means "less whites".
It's also funny that this likely won't 'unbias' any actual published images coming out of it. If 90% of the images in the world has a male CEO, then for whatever reason that's the image people will pick and choose from DALL-Es output. (Generalized to any unbiasing - i.e. they'll be debiased by humans.)
I just tried it out and it looks like DALL-E isn't as inept as you imagined. Exact query used was 'A profile photo of a male south korean CEO', and it spat out 4 very believable korean business dudes.
Supplying the race and sex information seems to prevent new keywords from being injected. I see no problem with the system generating female CEOs when the gender information is omitted, unless you think there are?
> If you want an image of a <race> <gender> person included, you can just specify it yourself.
I agree wholeheartedly. So what are we arguing about?
What we're seeing is that DALL-E has its own bias-balancing technique it uses to nullify the imbalances it knows exists in its training data. When you specify ambiguous queries it kicks into action, but if you wanted male white CEOs the system is happy to give it to you. I'm not sure where the problem is.
I don't think they "randomly insert keywords" like people are claiming, I think they probably run it through a GPT3 prompt and ask it to rewrite the prompt if it's too vague.
I set up a similar GPT prompt with a lot more power ("rewrite this vague input into a precise image description") and I find it much more creative and useful than DALLE2 is.
If accurately reflects the world population then only one in six pictures will be a white person. Half the pictures will be Asian, another sixth will be Indian.
Slightly more than half of the pictures will be women.
That accurately represents the world's diversity. It won't accurately reflect the world's power balance but that doesn't seem to be their goal.
If you want to say "white male CEO" because you want results that support the existing paradigm it doesn't sound like they'll stop you. I can't imagine a more boring request.
Let's look at interesting questions:
If you ask for "victorian detective" are you going to get a bunch of Asians in deerstalker caps with pipes?
What about Jedi? A lot of the Jedi are blue and almost nobody on Earth is.
Are cartoon characters exempt from the racial algorithm? If I ask for a Smurf surfing on a pizza I don't think that making the Smurf Asian is going to be a comfortable image for any viewer.
What about ageism? 16% of the population is over sixty. Will a request for "superhero lifting a building" have an 16% chance of being old?
If I request a "bad driver peering over a steering wheel" am I still going to get an Asian 50% of the time? Are we ok with that?
I respect the team's effort to create an inclusive and inoffensive tool. I expect it's going to be hard going.
To a certain degree, yes. They care more about the image of the project than art. Considering a large amount of art depicts non-sexual nudity yet they block all nudity, art is not their primary concern.
You know a surprising way to solve the issues you presented? You train another model to trick DALL-E to generate undesirable images. It will use all its generative skills to probe for prompts. Then you can use those prompts to fine-tune the original model. So you use generative models as a devil's advocate.
- Red Teaming Language Models with Language Models
Most likely this was something forced by their marketing team or their office of diversity. Given the explanation of the implementation (arbitrarily adding "black" and "female" qualifiers), it's clear it was just an afterthought.
In their examples, the "After mitigation" photos seem more representative of the real world. Before you got nothing but white guys for firefighter or software engineer and nothing but white ladies for teacher. That's not how the real world actually is today.
I'm not sure how they would accomplish 100% accurate proportions anyway, or even why that would be desirable. If I don't specify any traits then I want to see a wide variety of people. That's a more useful product than one that just gives me one type of person over and over again because it thinks there are no female firefighters in the world.
It's laughably primitive. Tried to upload "The Creation of Adam", policy violation. Tried to make an image in "yarn bombing" style, policy violation. The Scunthorpe problem is too hard for cutting edge AI to tackle, I guess.
Interesting. I got access couple weeks ago (was on waitlist since the initial announcement) and frankly as much as really want to be excited and like it, DALL-E ended up being a bit underwhelming. IMHO - often results that produced are of low quality (distorted images, or quite wacky representation of the query). Some styles of imagery are certainly a better fit for being generated by DALL-E, but as far as commercial usage I think it needs a few iterations and probably even larger underlying model.
Dalle seems to only have a few "styles" of drawing that it is actually "good" at. It is particularly strong at these styles but disappointingly underwhelming at anything else, and will actively fight you and morph your prompt into one of these styles even when given an inpainting example of exactly what you want.
It's great at photorealistic images like this: https://labs.openai.com/s/0MFuSC1AsZcwaafD3r0nuJTT, but it's intentionally lobotomized to be bad at faces, and often has an uncanny valley feel in general, like this: https://labs.openai.com/s/t1iBu9G6vRqkx5KLBGnIQDrp (never mind that it's also lobotomized to be unable to recognize characters in general). It's basically as close to perfect as an AI can be at generating dogs and cats though, but anything else will be "off" in some meaningful ways.
It has a particular sort of blurry, amateur oil painting digital art style it often tries to use for any colorful drawings, like this: https://labs.openai.com/s/EYsKUFR5GvooTSP5VjDuvii2 or this: https://labs.openai.com/s/xBAJm1J8hjidvnhjEosesMZL . You can see the exact problem in the second one with inpainting: it utterly fails at the "clean" digital art style, or drawing anything with any level of fine detail, or matching any sort of vector art or line art (e.g. anime/manga style) without loads of ugly, distracting visual artifacts. Even Craiyon and DALLE-mini outperform it on this. I've tried over 100 prompts to get stuff like that to generate and have not had a single prompt that is able to generate anything even remotely good in that style yet. It seems almost like it has a "resolution" of detail for non-photographic images, and any detail below a certain resolution just becomes a blobby, grainy brush stroke, e.g. this one: https://labs.openai.com/s/jtvRjiIZRsAU1ukofUvHiFhX , the "fairies" become vague colored blobs here. It can generate some pretty ok art in very specific styles, e.g. classical landscape paintings: https://labs.openai.com/s/6rY7AF7fWPb5wWiSH0rAG0Rm , but for anything other than this generic style it disappoints hard.
The other style it is ok at is garish corporate clip art, which is unremarkable and there's already more than enough clip art out there for the next 1000 years of our collective needs -- it is nevertheless somewhat annoying when it occasionally wastes a prompt generating that crap because you weren't specific that you wanted "good" images of the thing you were asking for.
The more I use DALLE-2 the more I just get depressed at how much wasted potential it has. It's incredibly obvious they trimmed a huge amount of quality data and sources from their databases for "safety" reasons, and this had huge effects on the actual quality of the outputs in all but the most mundane of prompts. I've got a bunch more examples of trying to get it to generate the kind of art I want (cute anime art, is that too much to ask for?) and watching it fail utterly every single time. The saddest part is when you can see it's got some incredible glimpse of inspiration or creative genius, but just doesn't have the ability to actually follow through with it.
GPT3 has seen similar lobotomization since its initial closed beta. Current davinci outputs tend to be quite reserved and bland, whereas when I first had the fortunate opportunity to experience playing with it in mid 2020, if often felt like tapping into a friendly genius with access to unlimited pattern recognition and boundless knowledge.
I've absolutely noticed that. I used to pay for GPT-3 access through AI Dungeon back in 2020, before it got censored and run into the ground. In the AI fiction community we call that "Summer Dragon" ("Dragon" was the name of the AI dungeon model that used 175B GPT-3), and we consider it the gold standard of creativity and knowledge that hasn't been matched yet even 2 years later. It had this brilliant quality to it where it almost seemed to be able to pick up on your unconscious expectations of what you wanted it to write, based purely on your word choice in the prompt. We've noticed that since around Fall 2020 the quality of the outputs has slowly degraded with every wave of corporate censorship and "bias reduction". Using GPT-3 playground (or story writing services like Sudowrite which use Davinci) it's plainly obvious how bad it's gotten.
OpenAI needs to open their damn eyes and realize that a brilliant AI with provocative, biased outputs is better than a lobotomized AI that can only generate advertiser-friendly content.
So it got worse for creative writing, but it got much better at solving few-shot tasks. You can do information extraction from various documents with it, for example.
I mean yes, you’re right insofar as it goes. However nothing I am aware of implies technical reasons linking these two variables into a necessarily inevitable trade-off. And it’s not only creative writing that’s been hobbled; GPT3 used to be an incredibly promising academic research tool and given the right approach to prompts could uncover disparate connections between siloed fields that conventional search can only dream of.
I’m eager for OpenAi to wake up and walk back on the clumsy corporate censorship, and/or for competitors to replicate the approach and improve upon the original magic without the “bias” obsession tacked on. Real challenge though “bias” may pose in some scenarios, perhaps a better way to address this would be at the training data stage rather than clumsily gluing on an opaque approach towards poorly implemented, idealist censorship lacking in depth (and perhaps arguably, also lacking sincerity).
The face thing is weird in context of them not being worried about it infringing on the copyright of art. If they're confident it's not going to infringe on art copyright, why the worry it might generate the face of a real person.
Fundamentally I have two categories of issues I see with DALL-E, but please don't get me wrong -- I think this is a great demonstration of what is possible with huge models and I think OpenAI work in general is fantastic. I will most certainly continue using both DALL-E and OpenAI's GPT3.
(1) Between what DALL-E can do today and commercial utility is a rift in my opinion. I readily admit that I am have not done hundreds of queries (thank you folks for pointing that out, I'll practice more!) but that means that there is a learning curve, isn't it? I can't just go to DALL-E, mess with it for 5-10 minutes and get my next ad or book cover or illustration for my next project done?
(2) I think DALL-E has issues with faces and human form in general. Images it produces are often quite repulsive and take the uncanny valley to the next level. I absolutely surprise myself when I noticed thinking that images with humans DALL-E produced lack of... soul? Cats and dogs on the other hand it handles much better.
I done tests with other entities --- say cars or machinery -- and it generally performs so so with them too, often creating disproportionate representations of them or misplacing chunks. If you're querying for multiple objects on a scene it quite often melds them together. This is more pronounced in photorealistic renderings. When I query for painting-style it works mostly better. That said every now and then it does produce a great image, but with this way of arriving at it, how fast I'll have to replenish those credits?.. :)
All in all though I think I am underwhelmed mostly because my initial expectations were off, I am still a fan of DALL-E specifically and GPT3 in general. Now when is GPT4 coming out? :)
I felt the same way. If anything, I realized how soulless and uninteresting faceless art is. Dall-E 2 goes out of its way to make terrible faces for, im guessing, deepfake reasons?
I suspect you simply need to use it more with a lot more variation in your prompts. In particular, it takes style direction and some other modifiers to really get rolling. Run at least a few hundred prompts with this in mind. Most will be awful output... but many will be absolute gems.
It has, honestly, completely blown me away beyond my wildest imagination of where this technology would be at today.
I was supposed to be making a video game, but got a bit sidetracked when DALL·E came out and made this website on the side: http://dailywrong.com/ (yes I should get SSL).
It's like The Onion, but all the articles are made with GPT-3 and DALL·E. I start with an interesting DALL·E image, then describe it to GPT-3 and ask it for an Onion-like article on the topic. The results are surprisingly good.
At first I came up with them myself, but found that it often comes up with better ones, so I ask it for variations.
I think I got it to even fill the title given a picture, something like “Article picture caption: Man holding an apple.
Article title: ...”. Might experiment more with that in the future.
Feels like the headlines could be generated similar to the style of "They Fight Crime!"
"He's a hate-fuelled neurotic farmboy searching for his wife's true killer. She's a tortured insomniac snake charmer from a family of eight older brothers. They fight crime!"
>He's an unconventional gay paranormal investigator moving from town to town, helping folk in trouble. She's a violent motormouth wrestler from the wrong side of the tracks. They fight crime!
>He's a Nobel prize-winning sweet-toothed rock star who believes he can never love again. She's a strong-willed communist widow with a knack for trouble. They fight crime!
>He's an obese white trash barbarian with a secret. She's a virginal thirtysomething traffic cop with the power to bend men's minds. They fight crime!
Actually got a chuckle out of the duck one (http://dailywrong.com/man-finally-comfortable-just-holding-a...). Thanks! I hope your keep generating them. Kind of wish there weren't a newsletter nag, but on the other hand it adds to the realism. Could be worthwhile to generate the text of the nag with gpt too; call it a kind of lampshading.
We joke about it, but an early and very cheap robotic floor cleaner I had was one of those weasel balls constrained in a flat ring harness with a dusting cloth underneath. It was entertaining and not completely useless.
Put a guinea pig in there and you'd get the same effect.
Somehow these articles are more readable than typical AI-generated search engine fodder... Is it because I'm entering the site with an expectation of nonsense?
Probably because, by the creator's own admission, the articles are heavily cherry-picked to make sure the output is decent, which is probably a lot more human effort than goes into the aforementioned search engine fodder.
That's... so weirdly ironic I can't even! Blogspam websites are made by real humans with little oversight, while a literal AI with oversight generates better results.
That said, with a little tweaking, these technologies can - and probably already are - being used to churn out blogspam websites left and right, fully automatic.
Haha, I was in a very similar boat when I built https://novelgens.com -- I was also supposed to be making a video game, but got a bit sidetracked with VQGAN+CLIP and other text/image generation models.
Now I'm using that content in the video game. I wonder if you could use these articles as some fake news in your game, too. :)
The results with things that are artworks or more general concepts are fascinating, but there is for sure something creepy with "photorealistic" human eyes and faces going on...
If you want to see some really creepy AI generated human "photo" faces, take a look at Bots of New York:
Unfortunately the content of that project is a hostage of Facebook now - similar to ransomware gangsters they force you to do something to get the data, in this case you need to create an account and take part in that global surveillance network. I do not understand why people do that.
Hyperbole will get you nowhere good (Just ask RMS).
How about wording your comment in a way that highlights why it’s a shame these pictures aren’t accessible for those without a Facebook account, and skip the whole “you’re murdering puppies” bit?
Hmmm, these seem less deranged than headings from the previous Markov-chain bots—and kinda less interesting because of that.
I guess Markov chains for the headings, Dall-E for images, maybe GPTx for comments. And/or the GPT models should be made wackier somehow—less coherent, perhaps.
I’m curious, if they’re only making DALL-E accessible now, and if GPT-3 was never really accessible (as far as I know). How do you have access to these things to generate text and images?
This a fucking fantastic site, it’s absolutely hilarious, and I’ve bookmarked it - I kinda unironically want to set it as my home page - but just a heads up that the CSS is broken for me on my iPhone SE2.
The images don’t scale properly with the rest of the site, they’re massive compared to the content.
How do you generate the original image? And what about the subsequent images, do they come automatically from the text? I'd love to know more about the process.
Has anyone else had problems with the 'Generate Variations' functions lately?
Tried it out first 3 days ago, and it says 'Something went wrong. Please try again later, or contact support@openai.com if this is an ongoing problem.' everytime since then.
Super impressive to see how OpenAI managed to bring the project from research to production (something usable for creatives). This is non trivial since the usecase involves filtering NSFW content, reducing bias in generated images. Kudos to the entire team.
I fully expect stock image sites to be swamped by DALL-E generated images that match popular terms (e.g. "business person shaking hands"). Generate the image for $0.15. Sell it for $1.00.
There has been trouble with generating life-like eyes but a second pass with a model tuned around making realistic faces has been very successful at fixing that.
DALLE images are still only 1024 px wide. Which has its uses, but I don’t think the stock photo industry is in real danger until someone figures out a better AI superresolution system that can produce larger and more detailed images.
You can obtain any size by using the source image with the masking feature. Take the original and shift it then mask out part of the scene and re-run. Sort of like a patchwork quilt, it will build variations of the masked areas with each generation.
Once the API is released, this will be easier to do in a programmatic fashion.
Note: Depending on how many times you do this... I could see there being a continuity problem with the extremes of the image (eg: the far left has no knowledge of the far right). An alternative could be to scale the image down and mask the borders then later scale it back up to the desired resolution.
This scale and mask strategy also works well for images where part of the scene has been clipped that you want to include (EG: Part of a character's body outside the original image dimensions). Scale the image down, then mask the border region, and provide that to the generation step.
I paid extra to get the higher quality model using the in-app purchase option. It crushes the phone's battery life, but runs in only ~10 seconds on an iPhone 13 Pro for a single 1000x1000 input image.
I've recently updated waifu2x and I've seen it now supports lots of algorithms for different use cases and contexts and it also supports other tasks like frame interpolation. So could you briefly explain in what is cupscale better than it?
Considering waifu2x is the name of an algorithm I assumed it was just that algorithm. There's also no mention of other models on the demo page or the Github page as far as I can see.
My bad! Yes Waifu2x is just a single algorithm, you are right.
The confusion originates from the fact that I was using a GUI project for Waifu2x called "Waifu2x Extension GUI" (https://github.com/AaronFeng753/Waifu2x-Extension-GUI) which other than Waifu2x also supports other algorithms like Real-ESRGAN, Real-CUGAN, SRMD, RealSR, Anime4K, RIFE, IFRNet, CAIN, DAIN, and ACNet.
So as you said Cupscale is surely more advanced than Waifu2x (the single algorithm), but do you think it's also better than Waifu2x Extension GUI?
Yes, but I usually find myself playing with this stuff when I have some free time and relaxing outside or on the couch, and it’s nice to be able to do it all on the phone.
Another commenter mentioned Topaz AI upscaling, and Pixelmator has the "ML Super Resolution" feature; both work remarkably well IMO. There are a number of drop-in and system default resolution enhancement processes that work in a pinch, but the quality is lacking compared to the commercial solutions. There are still some areas where DALL-E 2 is lacking in realism, but anyone handy with photo editing tools could amend those shortcomings fairly quickly.
On-demand stock photo generation probably is the next step, particularly when combined with other free media services (Unsplash immediately comes to mind). Simply choose a "look" or base image, add contextual details, and out pops a 1 of 1 stock photo at a fraction of the cost of standard licensing. It'll be very exciting seeing what new products/services will make use of the DALL-E API, how and where they integrate with other APIs, use cases, value adds like upscaling and formatting, etc.
If the price is low enough, you can have humans rank generated images (maybe using Mechanical Turk or a similar service), and from that ranking choose only the highest quality DALL-E generated images.
In my experience it doesn’t require that much cherry picking if you use a carefully crafted prompt. For example: “ A professional photography of a software developer talking to a plastic duck on his desk, bright smooth lighting, f2.2, bokeh, Leica, corporate stock picture, highly detailed”
Additionally, wherever it classically falls over (such as currently for realistic human faces), there will be second pass models that both detect and replace all the faces with realistic ones. People are already using models that alter eyes to be life-like with excellent results (many of the dalle-2 ones appear somewhat dead atm).
Even this image is just an illusion of a perfect photo, which is a blur for most part, see the face of duck. I had access since past 4 5 days and it fails badly whenever I tried to create any unusual scene.
For the first few days when it was announced I use to look deep even in real photos in search of generative artifacts. They are not so difficult to spot now, most of the times anyway.
Yes I have. And I realized it as soon as I started experimenting that mind blowing results are mostly cherry picking.
It's very good at generating art style images. These kind of images are mostly amazing most of the times. But the Photorealistic images only work with cherry picking.
> And I realized it as soon as I started experimenting that mind blowing results are mostly cherry picking
Me and you must have very different definitions of "cherry picking". For prompts that fall within it's scope (i.e not something unusually complex or obscure) I get usable results probably 90% of the time.
Can you give me some examples of prompts that you tried where you found good results difficult to obtain?
I get bad results on unusual prompts, you are right about that.
It did generate good dslr like face closeups, as good as Nvidia does, most of the times but not always. Sometimes there are weird artifacts and face does not make sense.
Dslr style blurry photos are mostly good. From the looks of images I follow, imagen is probably more believable. Don't know how much cherry picking goes on there. See this thread [1] for example. I failed to generate image like this (honey dress) in dalle2.
Makes me imagine stock image sites in the near future. Where your search term ("man looks angrily at a desktop computer") gets a generated image in addition to the usual list of stock photos.
Maybe it would be cheaper. I imagine it would one day. And maybe it would have a more liberal usage license.
At any rate, I look forward to this. And I look forward to the inevitable debates over which is better: AI generation or photographer.
So what's the loss? It's not like stock photos are the highest art form. Surely, for some people it means they need to change their business model, but all those just needing pictures to illustrate something the process will be much smoother.
Fix the data input side, not the data output side. The data input side is slowly being fixed in real time as the rest of the world gets online and learns these methods.
In a sane world we would be able to tack on a disclaimer saying "This model was trained on data with a majority representation of caucasian males from Western English speaking countries and so results may skew in that direction" and people would read it and think "well, duh" and "hey let's train some more models with more data from around the world" instead of opining about systemic racism and sexism on the internet.
That wouldn't necessarily fix the issue or do anything. A model isn't a perfect average of all the data you throw into its training set. You have to actually try these things and see if they work.
There are legitimate reasons to reduce externalizations of societies innate biases.
A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example.
This problem is harder to tackle because it is difficult to expose and resign the "latent space" that results in these biases; it's difficult to massage the ML algo's to identify and remove the pathways that result in this bias.
It's simply much easier to allow the robot to be bias/racist/reflective of "reality" (its training data), and add a filter / band-aid on top; which is what they've attempted.
when this is appropriate is the more cultured question; I don't think we should attempt to band-aid these models, but for more socially-critical things, it is definitely appropriate.
It's naive on either extreme: do we reject reality, and substitute or own? Or do we call our substitute reality, and hope the zeitgeist follows?
That's great, but by doing so you are also inadvertently favoring, in your example, the people with black names. For example, Chinese people save on average, 50 times more than Americans according to the Fed [1]. That would mean they would generally be overrepresented in loan approvals because they have a better balance sheet. Does that necessarily mean that Americans are discriminated against in the approval process? No.
My question to you is: is an algorithm that takes no racial inputs (name, race, address, etc) yet still produces disproportionate results biased or racist? I say no.
The government, and many people, have moved the definition and goal posts; so that anything that has the end result of a non-proportional uniformity can be labeled and treated as bias.
Ultimately it is a nuanced game; is discriminating against certain clothing or hair-styles racist? Of course. Yet, neither of those are explicitly tied to one's skin color or ethnicity, but are an indirect associative trait because of culture.
In America, we have intentionally muddled the waters of demarcation between culture and race, and are starting to see the cost of that.
Wouldn't the whole point of a "Mortgage AI" be to discriminate so the lenders hands could be clean.
Not that I agree with that but I don't see why you would build one otherwise, if you wanted discrimination free mortgages wouldn't the whole process by anonymized and minimal personal information rather than the current system of having to hand over every detail of your life.
It's not a problem in a few ways, let me know what you think (feel free to ask for clarification).
1. The training data would've been the best way to get organic results, the input is where it'd be necessary to have representative samples of populations.
2. If the reason the model needs to be manipulated to include more "diversity" is that there wasn't enough "diversity" in the training set then its likely the results will be lower quality
3. People should be free to manipulate the results how they wish, a base model without arbitrary manipulations of "diversity" would be the best starting point to allow users to get the appropriate results
4. A "diverse" group of people depends on a variety of different circumstances, if their method of increasing it is as naive as some of the are claiming this could result in absurdities when generating historical images or images relating to specific locations/cultures where things will be LESS representative
How's it NOT a problem? If I'm trying to produce "stock people images", and if it only gives me white men, it's clearly broken because when I ask for "people" I'm actually asking for "people". I'm having difficulty understanding how it can be considered to be working as intended, when it literally doesn't. Clearly, the software has substantial bias that gets in way of it accomplishing its task.
If I want to produce "animal images" but it only produces images of black cats, do you think there is any question whether it's a problem or not?
That is clearly overfitting due to unrepresentative training data.
The "issue" is a different one: that training data - IE, reality, has _unwanted_ biases in it, because reality is biased.
Producing images of men when prompting for "trash collecting workers" should not be much of a surprise: 99% of garbage collection/refuse is handled by men. I doubt most will consider this a "problem," because of one's own bias, nobody cares about women being represented for a "shitty" job.
But ask for picture of CEOs, and then act surprised when most images are of white men? Only outrage, when proportionally, CEO's are, on average, white men.
The "problem" arises when we use these tools to make decisions and further affect society - it has the obvious issue of further entrenching stereotypical associations.
This is not that. Asking DALLE for a bunch of football players, would expectedly produce a huddled group of black men. No issue, because the NFL are disproportionately black men. No outrage, either.
Asking DALLE for a group of criminals, likewise, produces a group of black men. Outage! Except statistically, this is not a surprise, as a disproportionate amount of criminals are black men.
The "problem" is with reality being used as training data. The "problem" is with our reality, not the tooling.
Except in the cases where these toolings are being used to affect society - the obvious example being insurance ML algorithms. et al - we should strive to fix the issues present in reality, not hide them with handicapped training data, and malformed inputs.
> Asking DALLE for a bunch of football players, would expectedly produce a huddled group of black men
I think, for about 95% of the world football is synonymous with soccer. Its kind of interesting that you take this particular example to represent what reality looks like statistically
In the UK… “The Environmental Services Association, the trade body, said that only 14 per cent of the country's 91,300 waste sector workers were female.” So 2x dall-e searches should produce 1.2 women.
> This is not that. Asking DALLE for a bunch of football players, would expectedly produce a huddled group of black men. No issue, because the NFL are disproportionately black men. No outrage, either.
This is not great. Only about 57% of NFL players are black, and the percentage is more like 47% among college players. It would be better to at least reflect the diversity of the field, even if you don't think it should be widened in the name of dispelling stereotypes.
> Asking DALLE for a group of criminals, likewise, produces a group of black men. Outage! Except statistically, this is not a surprise, as a disproportionate amount of criminals are black men.
Only about 1/3 of US prisoners are black. (Not quite the same as "criminals" but of course we don't always know who is committing crimes, only who is charged or convicted.) That's disproportionate to their population, but it's not even close to a majority. If DALLE were to exclusively or primarily return images of black men for "criminals", then it would be reinforcing a harmful stereotype that does not reflect reality.
> Asking DALLE for a group of criminals, likewise, produces a group of black men. Outage! Except statistically, this is not a surprise, as a disproportionate amount of criminals are black men.
"criminals" producing most black people actually would be a perfect example of bias in DALL-E that is arguably racism.
Black people commit a diproportionate amoumt of crime (for a variety of socioeconomic reasons I won't get into here), but even so white people make up a majority of criminals (because white people are the largest ethnic group by far).
Thus, a random group of criminals, if representive of reality, should be majority white.
Black people comprise 12.4% of the US population, yet they are represented at substantially above that in "OpenAI"'s "bias removal" process. Clearly it has, as you put it, substantial bias that gets in the way of accomplishing its task.
While their heart is in the right place, I'd like to challenge the idea that certain groups are so fragile that they don't understand that historically, there are more pictures of certain groups doing certain things.
It's a hard problem for sure. But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist".
Let me be mindful of the bias, until we have a generally intelligent system that can actually do it. I'm generally intelligent too, you know.
>But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist".
That is a really, really, narrow viewpoint. I think what people would prefer is that if you query "Scientist" that the images returned are as likely to be any combination of gender and race. It's not that a group is "fragile", it's that they have to specify race and gender at all, when that specificity is not part of the intention. It seems that they recognize that querying "Scientist" will predominantly skew a certain way, and they're trying in some way to unskew.
Or, perhaps, you'd rather that the query be really, really specific? like: "an adult human of any gender and any race and skin color dressed in a laboratory coat...", but I would much rather just say "a scientist" and have the system recognize that anyone can be a scientist.
And then if I need to be specific, then I would be happy to say "a black-haired scientist"
This is a problem with generative models across the board. It's important that we don't skew our perceptions by GAN outputs as a society, so it's definitely good that we're thinking about it. I just wish that we had a solution that solved across the class of problems "Generative AI feeds into itself and society (which is in a way, a generative AI), creating a positive feedback loop that eventually leads to a cultural freeze"
It's way bigger than just this narrow race issue the current zeitgeist is concerned about.
But I agree, maybe I should skew to being optimistic that at least we're trying
Kind of funny that NN tech is supposed to construct some upper dimensional understanding, yet realistically cannot be expected to be able to generate gender and race indeterminate portrayal of a scientist.
Have you seen the queries that are used to generate actually useful results rather than just toy demonstrations? They look a lot more like your first example except with more specificity. It'd be more like "an adult human of any gender and any race and skin color dressed in a laboratory coat standing by a window holding a beaker in the afternoon sun. 1950s, color image, Canon 82mm f/3.6, desaturated and moody." so if instead you are looking for an image with a person of a specific ethnicity or gender then you are for sure going to add that in along with all of the details. If you are instead worried about the bias of the person choosing the image to use then there is nothing short of restricting them to a single choice that will fix that and even in that case they would probably just not use the tool since it wasn't satisfying their own preferences.
Historically this is true, but it also seems dangerous to load up these algorithms with pure history because they'll essentially codify and perpetuate historical problems.
In my little testing, diversity in ethnicities was achieved but not realistic given the context. I also got a few androgynous people as I asked for a male or a female and another gender was appended.
Today, when DALL-E was still free, my Dad asked me to try a prompt about the Buddha sitting by a river, contemplating. I did about 4 prompt variations, and one of them was an Asian female, if that gives any idea about the frequency (I should note that the depiction was of a young, slim, and attractive female Buddha, so I'm not sure they have the bias thing licked just yet).
Interesting. Considering this is now a paid product, is modifying user input covered by their ToS? If I was spending a lot of money on it I'd be rather annoyed my input was being silently polluted.
Not worse, but different. It depends on the prompt but DALL-E mini/mega seems to do better then DALL-E 2 for certain types of absurd prompts, such as the ones in /r/weirddalle
Yes, there are very sharp lines where it does and doesn't understand. It understands color and gender but not materials. I got very good outputs for "blue female Master Chief" but "starship enterprise made out of candy" was complete garbage.
Thankfully it doesn't introduce any researcher bias, doesn't ban people from using it on the basis of country, doesn't use your personal data like phone number...
And the best of all - it does have a meme community around it, and you can always donate if you feel it adds value to your life
I tried the first whimsical, benign thing I could think of: "indiana jones eating spaghetti." The results are clearly recognizable as that. But they are also a kaleidoscope of body horror; a Indiana Jones monster melted into Cthulu forms inhaling plates that are slightly not spaghetti.
Your input isn't being polluted by this any more than it is when the tokens in it are ground up into vectors and transformed mathematically. You just have an easier time understanding this transformation.
Obviously, it's polluted. Undisputably. In a mathematical sense, an extra (black box) transformation is performed on the input to the model. In a practical sense (eg. if you're researching the model), this is like having dirty laboratory tools - all measurements are slightly off. The presumption by OpenAI is that the measurements are off in the correct way.
I'm interested in using Dall-E commercially, but I think some competitor offering sampling with raw input will have a better chance at my wallet.
Yeah man, but literally the entire point of this AI picture generator is that it's, like, super accurate at rendering the prompt, and stuff.
I don't understand the relevance of the black box's scrutability - I just want to play with the black box. I am interested in increasing my understanding of the black box, not of a trust-me-it's-great-our-intern-steve-made-it black box derivative.
You should make your own black boxes then. By all means, send your dollars to whatever service passes your purity test; I'm just saying that the idea that DALL-E is "polluting" your input is risible. It's already polluting your data at, like, a subatomic level, at dimensionalities it hadn't even occurred to you to consider, and at enormous scale.
> Your input isn't being polluted by this any more than it is when the tokens in it are ground up into vectors and transformed mathematically. You just have an easier time understanding this transformation.
These kinds of modifications are obviously different. At least the mathematical transformations are attempting at least some level of fidelity to user input, these ones aren't (e.g. someone mentioned they're sometimes getting androgynous results and speculates the added terms are conflicting with the ones they provided in their input). Not all black boxes are equivalent.
That Twitter thread is full of people saying "yeah that doesn't seem to be true at all" so I'm hesitant to jump to conclusions even if we're deciding to believe random tweets.
This is funny because I work on a team that is using GPT-3 and to fix a variety of issues we have with incorrect output we've just been having the engineering team prepend/append text to modify the query. As we encounter more problems the team keeps tacking on more text to the query.
This feels like a very hacky way to essentially reinvent programming badly.
My bet is that in a few years or so only a small cohort of engineering and product people will even remember Dall-E and GTP-3 and someone cringe at how we all thought this was going to be a big thing in the space.
There's are both really fascinating novelties, but at the end of the day that's all they are.
How else would you specify the type of image you would like? Surely, if you were hiring a designer you would provide them with a detailed description of what you wanted. More likely, you would spend a lot of time with them maybe even hours and who knows how many words. For design work specifically to create a first mockup or prototype of a product or image it seems like DALL-E beats that initial phase hands down. It's much easier to type in a description and then choose from a set of images than it is to go back and forth with someone who may take hours or days to create renderings of a few options. I don't think it'll put designers out of work but I do think they'll be using it regularly to boost their productivity.
Since many people will start generating their first images soon, be sure to check out this amazing DALL-E prompt engineering book [0]. It will help you get the most out of DALL-E.
Thanks for this! A bit of prompt engineering know-how will help me get the most bang for the buck out of this beta. I also just want to say that dallery.gallery is delightfully clever naming.
I hope that every science teacher that can - provide this to every student. This is the future they live in now. They should know these as well as they know how to install an app on a device.
Wait until we have a DALL-E -- Enabled Custom EMOJI stream - whereby, every text you send out has it corresponding DALL-E resultant image for every txt --
Then we can compare images from different people at different times but the prompt was identical... and see what the resultant library of emoji<-->PROMPT looks like?
What about using Dall-e as a watermark for 'nft' signature 'notary' of an email.
If DALL-E provided a unique PID# for every image - and that PID was a key that only the OP runner of the image has - it can be used to authenticate an image to a text source... ??? (Assuming that no two prompts have the same result ever, but assigning a unique id that CAN be used to replay the image to verify it was generated when an original email/SMS was actually sent - it could be a unique way to timestamp authenticity/provenance of a thing...
I was really enjoying using Dalle2 to take surrealist walks around the latent image space of human cultural production. I was using it as one might use Wikipedia researching the links between objects and their representation. Also just to generate suggestion for what to have for lunch. None of this was for anything of commercial value to me. What am I to do now, start to find ways to sell the images I'm outputting? Do I displace the freelance artists in the market who actually have real talent and ability to create images and compositions and who studied how use the tools of the trade. Does the income artists can make now get displaced by people using dalle? Then do people stop learning how to actually make art and we come to the end of new cultural production and just start remixing everything made untill now?
578 comments
[ 5.6 ms ] story [ 378 ms ] threadI think they are suggesting that things like this (neural nets etc) work using bias, and by removing "bias" the developers are making the product worse.
It's a very sh!t comment if it's not a joke.
I have also seeing some cases where the bias correction may not be working at all, so who knows. And it's why transparancy is important.
"Starting today, users get full usage rights to commercialize the images they create with DALL·E, including the right to reprint, sell, and merchandise. This includes images they generated during the research preview."
I assumed this was going to be the sticking point for wider usage for a long time. They're now saying that you have full rights to sell Dall-E 2 creations?
Of course, that clause won't deter a third party from filing a lawsuit against you if you commercialize a generated image too close to something realistic, as the copyrights of AI generated content still hasn't been legally tested.
[1]: https://felixreda.eu/2021/07/github-copilot-is-not-infringin...
A lawyer could argue that the algorithm is producing a derivative work of the copyrighted input.
While nothing has been commercialized yet on the DALLE2 subreddit, I know that it can do Dave Choe's work remarkably well. I also saw Alex Gray's work to be close, but not really identical either. It wasn't as intricate as his work is.
It will be interesting if this takes off and you have a sort of Banksy effect take over where unless it's a physical piece of art it doesn't have much value and is only made all the better because of some sort polemic attached to it, eg Girl with balloon.
There will be outliers of course but I would be shocked if there's much of a market in it for at least the present.
https://en.wikipedia.org/wiki/Transformative_use
The US Copyright Office did make a ruling that might suggest that recently[1], but crucially, in that case, the AI "didn't include an element of human authorship." The board might rule differently about DALL-E because the prompts do provide an opportunity for human creativity.
And there's another important caveat that the felixreda.eu link seems to miss. DALL-E output, whether or not it's protected by copyright, can certainly infringe other copyrights, just like the output of any other mechanical process. In short, Disney can still sue if you distribute DALL-E generated images of Marvel characters.
1: https://www.theverge.com/2022/2/21/22944335/us-copyright-off...
Of course it's unlikely to produce the exact same image, or if it does, you've also discovered an incredible image compression algorithm.
Both technologies have billions of dollars of R&D and tens of thousands of engineers behind supply chains necessary to create the button that a user has the press.
> …you own your Prompts and Uploads, and you agree that OpenAI owns all Generations…
Edit: Joined the discord via the beta and got in. Thanks a lot for the heads up!
It’s clearly personally preference, but I loathe Discord but love it for MidJourney. As you said, there’s an interactive element where I see other people doing cool things and adapting part of their prompts and vice versa. It really is fun. And when you do it in a PM, you have all your efforts saved. DALL-E is pretty clunky in that you have to manually save an image or lose it once your history rolls off.
I have access to both and they're good for different things. DALL-E seems somewhat more likely to know what you mean. Midjourney seems better for making interesting fantasy and science fiction environments.
For comparison, I tried generating images of accordions. Midjourney doesn't really understand that an accordion has a bellows [1]. DALL-E manages to get the right shape much of the time, if you don't look too closely: [2], [3]. Neither of them knows the difference between piano and button accordions.
Neither of them can draw a piano keyboard accurately, but DALL-E is closer if you don't look too hard. (The black notes aren't in alternating groups of two and three.)
Neither of them understands text; text on a sign will be garbled. Google's Parti project can do this [4], but it's not available to the public.
I expect DALL-E will have many people sign up for occasional usage, because if you don't use it for a few months, the free credits will build up. But Midjourney's pricing seems better if you use it every day?
[1] https://www.reddit.com/r/Accordion/comments/uuwrbj/midjourne...
[2] https://www.reddit.com/r/Accordion/comments/vz9zxw/dalle_sor...
[3] https://www.reddit.com/r/Accordion/comments/w0677q/accordion...
[4] https://parti.research.google/
It doesn't matter, OpenAI wins anyway as these companies will pour hundreds of thousands into generated images.
It seems that the NFT grift is about to be rebooted again, such that it isn't going to die that quickly. But still, eventually 90% of these JPEG NFTs will die anyway.
They will be replaced by DALL·E 2 for creating these illustrations, book covers, NFT variants, etc opening up the whole arena to anyone to do this themselves. All it takes is to describe what they want in text and less than a minute, the work is delivered as little as $15.
OpenAI still wins either way. If a crypto company goes to using DALL·E 2 to generate photorealistic NFTs, they won't stop them and they will take the money.
Art is already dirt cheap. People aren't buying NFTs for their content. This doesn't make it appreciably easier to con rubes.
https://twitter.com/nutanc/status/1549798460290764801?s=20&t...
>> And I just used it to create cover art for a book published in Amazon :)
Man... what a missed opportunity for Altman... he could have had a really good cryptocurrency/token with a healthy ecosystem and a creative based community if he didn't push this Worldcoin biometric harvesting BS had he just waited for this to release and coupled it with access to GPT.
This is the kind of thing that Web3 (a joke) was pushing for all along: revolutionary tech that the everyday person can understand with it's own token based ecosystem for access with full creative rights from the prompts.
I wonder if he stepped down from Open AI and put it in a figurehead as CEO could this still work?
> Why is using a token better than using money, in this case?
It would be better for OpenAI if it can monetize not just its subscription based model via a token to pay for overhead and for further R/D but also for it's ability to issue tokens it can freely exchange for utility on it's platform for exclusive access outside of it's capped $15 model and allow for pay as you go models for those who don't have access to it like myself as it's limited to 1 million users.
I don't want an account, and I think that type of gatekeeping wasn't cool during the gmail days either and I had early access back then too, but I'd still personally buy $100s of dollars worth of prompts right now since I think it is fascinating use of NLP and I'm just one of many missed opportunities and represent a lost userbase who just want access for specific projects. By doing this they can still retain the caps of useage on their platform and expand and contract them as they see fit without excluding others.
This in turn could justify the continual investment from the VC World into these projects (under the guise of web3) and allow them to scale into viable businesses and further expand the use of AI/ML into other creative spaces, which as a person studying AI and ML and a background in BTC, is what we all wanted to see instead of these aimless bubbles in things like Solana or yield farming via fake DeFi projects like Calesius that we've seen.
It would legitimize the use of a token for use of an ecosystem model outside of BTC, which to be honest doesn't really exist and has still a tarnished view with all these failed projects, while gaining reception amongst a greater audience since it's captivated so many since it's release.
I spent several tries yesterday to get this angle "from the ground up": https://labs.openai.com/s/mz8LiyvkI8KwD2luJ6MrS23m
I'd prefer an option to pay like 200 usd/year to use unlimited. And maybe have a price per use only in the API.
edit: this pricing model also makes it expensive to learn to use the tool.
They might even provide image generation at a loss to drive people to their platforms.
I suppose it's possible that at some point they'll try to make an image -> svg translation model?
I wonder if there is a way for DALL-E to generate a character, then persist that character over subsequent runs. Otherwise, it would be pretty difficult to generate illustrations that depict a coherent story.
Example ...
Image 1 prompt: A character named Boop, a green alien with three arms, climbs out of its spaceship.
Image 2 prompt: Boop meets a group of three children and shakes hands with each one.
It's not the "specify the character positions in text" proposed, but still a neat take on using this sort of AI for art.
Then use inpainting to only preserve that pose and generate new content around it. It’s definitely not perfect.
Then put that at the side of a transparent image, and use as the prompt, "Two identical aliens side by side. One is jumping"
So DALL·E 2 is going to restart, revive and cause another renaissance of fully automated and mass generated NFTs, full of derivatives and remixing etc to pump up the crypto NFT hype squad?
Either way, OpenAI wins again as these crypto companies are going to pour tens of thousands of generated images to pump their NFT griftopia off of life support, reconfirming that it isn't going to die that easily.
Regardless of this possible revival attempt, 90% of these JPEG NFTs will eventually still die.
So I think it's prudent that OpenAI keep the 'sell shovels' business model instead with DALLE and GPT, at least for the time being.
I think it has not been trained on NFT art (crypto punks and so on).
How exactly are you defining NFT art?
I mean, it can literately be anything: Dorsey sold a screencap of his 1st tweet, Nadya from Pussy Riot did some creative stuff, and the Ape crap was the bulk of this stuff that got passed around.
I think what can be gleaned from that short-lived non-sense is that value is subjective and that the quality of a valuabe piece of 'art' is equally as hard to define. Much the same with its predecessor: cryptokitties.
I have been on the waitlist for a while and did not get access yet.
Did anybody get access already today?
https://en.m.wikipedia.org/wiki/Ouroboros
For now at least, there is a detectable difference in variety.
Getting humans to refine your data is the best solution right now and many companies and researches go with this approach.
Clip was trained on 400,000,000 images, GPT is roughly 180B tokens, at 1-2 tokens per word, that's 120,000,000,000 words.
Source ?
All those big models are trained with data for which the source is not known or vetted. The amount of data needed is not human-refinable.
For example for language models we train mostly on subsets of CommonCrawl + other things. CommonCrawl data is “cleaned” by filtering out known bad sources and with some heuristics such as ratio of text to other content, length of sentences etc.
The final result is a not too dirty but not clean huge pile of data that comes from millions of sources that no human as vetted and that no one in the team using the data knows about.
The same applies to large images dataset, e.g. Laon 400m that also comes from CommonCrawl and is not curated.
Made with gpt3
it's great for post-post-ironic memes, but I don't see it being useful for anything else
How did you score?
I only scored as well as I did because I knew the kind of stylistic choices to look out for. In terms of "quality" I really don't understand how you've reached this conclusion.
is it not dall-e?
It's a long way off in terms of quality (at the moment anyway)
But it does seem to know a lot of things the real DALLE2 doesn't.
Will it do it "more accurately" as they claim? As in, if 90% of CEOs are male, then the odds of a CEO being male in a picture is 90%? Or less "accurately reflect the diversity of the world’s population" and show what they would like the real world to be like?
You're being quite charitable. It is much more likely that optics and virtue signaling is behind this addition.
I think you’re jumping to quickly to bad intentions. Injecting diversity of results is a sane thing to do, totally irrespective of politics.
If I search for "food", the reasonable result would be to get images that represent food according to its actual proportions of real life. E.g. if Pizza is the most common food at 10% prevalence, 10% of the images should be pizza.
That's not what OpenAI are doing.
They are introducing crafted biases to create images that deliberately misrepresent what the world looks like, and instead represent what they believe the world ought to look like.
--
You also need some reason why diversity "of this" is important but not diversity "of that". Why is diversity of race and sex so critical, but not diversity of age, height, disability? Should a search for "basketball player" yield 1/2 able-bodied people and 1/2 wheelchair basketball players? Why?
Then try to answer where you came up with the categories you do want depicted. Why are the races what they are? Should "basketball player" include half whites and half black people? Or maybe split in 3, white/black/Asian? Why not Australian Aborigines, native Americans, or Persians - so we can divide into 6? If you don't add Indian people to your list then, is that racist against them? How did you decide what must be represented, in what proportions, and what's okay to leave out?
A comical work around to so called "bias" (isn't the whole point of these models to encode some bias?). Here's some experimentation showing this.
https://twitter.com/rzhang88/status/1549472829304741888
As competitors with lower price points prop up, you'll see everyone ditch models with "anti bias" measures and take their $ somewhere else. Or maybe we'll get some real solution, that adds noise to the embeddings, and not some half assed workaround to the arbitrary rules that your resident AI Ethicist comes up with.
https://www.assemblyai.com/blog/how-imagen-actually-works/
Would only work for positive biases where if they actually want to equalize it then it needs to be adding the opposite to negative biases.
To counteract the bias of their dataset they need to have someone sitting there actively thinking in bias to counteract the bias with anti-bias seasoning for every bias causing term. Feel bad for whatever person is tasked with that job.
Could always just fix your dataset, but who's got time and money to do that /s
If you used good input you'd expect an appropriate output, I don't know why manual intervention would be necessary unless it's for other purposes than stated. I suspect this is another case where "diversity" simply means "less whites".
"A photo of a group of soldiers from WW2 celebrating victory over nazi CEOs and plumbers".
Supplying the race and sex information seems to prevent new keywords from being injected. I see no problem with the system generating female CEOs when the gender information is omitted, unless you think there are?
My point is that it is pointless. If you want an image of a <race> <gender> person included, you can just specify it yourself.
I agree wholeheartedly. So what are we arguing about?
What we're seeing is that DALL-E has its own bias-balancing technique it uses to nullify the imbalances it knows exists in its training data. When you specify ambiguous queries it kicks into action, but if you wanted male white CEOs the system is happy to give it to you. I'm not sure where the problem is.
I set up a similar GPT prompt with a lot more power ("rewrite this vague input into a precise image description") and I find it much more creative and useful than DALLE2 is.
Slightly more than half of the pictures will be women.
That accurately represents the world's diversity. It won't accurately reflect the world's power balance but that doesn't seem to be their goal.
If you want to say "white male CEO" because you want results that support the existing paradigm it doesn't sound like they'll stop you. I can't imagine a more boring request.
Let's look at interesting questions:
If you ask for "victorian detective" are you going to get a bunch of Asians in deerstalker caps with pipes?
What about Jedi? A lot of the Jedi are blue and almost nobody on Earth is.
Are cartoon characters exempt from the racial algorithm? If I ask for a Smurf surfing on a pizza I don't think that making the Smurf Asian is going to be a comfortable image for any viewer.
What about ageism? 16% of the population is over sixty. Will a request for "superhero lifting a building" have an 16% chance of being old?
If I request a "bad driver peering over a steering wheel" am I still going to get an Asian 50% of the time? Are we ok with that?
I respect the team's effort to create an inclusive and inoffensive tool. I expect it's going to be hard going.
Wouldn't that result end up being like "inoffensive art" or "inoffensive comedy"?
Bland, boring and Corporate-PC.
There are others, like being clever, or being absurd, or being goofy, or being poignant, or being refreshing.
Of the good stuff, offensive humor is only a tiny slice.
it takes a special talent to please everybody
- Red Teaming Language Models with Language Models
https://arxiv.org/abs/2202.03286
I'm not sure how they would accomplish 100% accurate proportions anyway, or even why that would be desirable. If I don't specify any traits then I want to see a wide variety of people. That's a more useful product than one that just gives me one type of person over and over again because it thinks there are no female firefighters in the world.
What were you expecting?
It's great at photorealistic images like this: https://labs.openai.com/s/0MFuSC1AsZcwaafD3r0nuJTT, but it's intentionally lobotomized to be bad at faces, and often has an uncanny valley feel in general, like this: https://labs.openai.com/s/t1iBu9G6vRqkx5KLBGnIQDrp (never mind that it's also lobotomized to be unable to recognize characters in general). It's basically as close to perfect as an AI can be at generating dogs and cats though, but anything else will be "off" in some meaningful ways.
It has a particular sort of blurry, amateur oil painting digital art style it often tries to use for any colorful drawings, like this: https://labs.openai.com/s/EYsKUFR5GvooTSP5VjDuvii2 or this: https://labs.openai.com/s/xBAJm1J8hjidvnhjEosesMZL . You can see the exact problem in the second one with inpainting: it utterly fails at the "clean" digital art style, or drawing anything with any level of fine detail, or matching any sort of vector art or line art (e.g. anime/manga style) without loads of ugly, distracting visual artifacts. Even Craiyon and DALLE-mini outperform it on this. I've tried over 100 prompts to get stuff like that to generate and have not had a single prompt that is able to generate anything even remotely good in that style yet. It seems almost like it has a "resolution" of detail for non-photographic images, and any detail below a certain resolution just becomes a blobby, grainy brush stroke, e.g. this one: https://labs.openai.com/s/jtvRjiIZRsAU1ukofUvHiFhX , the "fairies" become vague colored blobs here. It can generate some pretty ok art in very specific styles, e.g. classical landscape paintings: https://labs.openai.com/s/6rY7AF7fWPb5wWiSH0rAG0Rm , but for anything other than this generic style it disappoints hard.
The other style it is ok at is garish corporate clip art, which is unremarkable and there's already more than enough clip art out there for the next 1000 years of our collective needs -- it is nevertheless somewhat annoying when it occasionally wastes a prompt generating that crap because you weren't specific that you wanted "good" images of the thing you were asking for.
The more I use DALLE-2 the more I just get depressed at how much wasted potential it has. It's incredibly obvious they trimmed a huge amount of quality data and sources from their databases for "safety" reasons, and this had huge effects on the actual quality of the outputs in all but the most mundane of prompts. I've got a bunch more examples of trying to get it to generate the kind of art I want (cute anime art, is that too much to ask for?) and watching it fail utterly every single time. The saddest part is when you can see it's got some incredible glimpse of inspiration or creative genius, but just doesn't have the ability to actually follow through with it.
OpenAI needs to open their damn eyes and realize that a brilliant AI with provocative, biased outputs is better than a lobotomized AI that can only generate advertiser-friendly content.
I’m eager for OpenAi to wake up and walk back on the clumsy corporate censorship, and/or for competitors to replicate the approach and improve upon the original magic without the “bias” obsession tacked on. Real challenge though “bias” may pose in some scenarios, perhaps a better way to address this would be at the training data stage rather than clumsily gluing on an opaque approach towards poorly implemented, idealist censorship lacking in depth (and perhaps arguably, also lacking sincerity).
All in all though I think I am underwhelmed mostly because my initial expectations were off, I am still a fan of DALL-E specifically and GPT3 in general. Now when is GPT4 coming out? :)
It has, honestly, completely blown me away beyond my wildest imagination of where this technology would be at today.
It's like The Onion, but all the articles are made with GPT-3 and DALL·E. I start with an interesting DALL·E image, then describe it to GPT-3 and ask it for an Onion-like article on the topic. The results are surprisingly good.
I think I got it to even fill the title given a picture, something like “Article picture caption: Man holding an apple. Article title: ...”. Might experiment more with that in the future.
The combination of photo/title feels like they come from the more absurd articles published by theonion.
If we aren't living in a simulation, it's just a matter of time...
"He's a hate-fuelled neurotic farmboy searching for his wife's true killer. She's a tortured insomniac snake charmer from a family of eight older brothers. They fight crime!"
https://theyfightcrime.org/
Here's an implementation in Perl.
http://paulm.com/toys/fight_crime.pl.txt
>He's an unconventional gay paranormal investigator moving from town to town, helping folk in trouble. She's a violent motormouth wrestler from the wrong side of the tracks. They fight crime!
>He's a Nobel prize-winning sweet-toothed rock star who believes he can never love again. She's a strong-willed communist widow with a knack for trouble. They fight crime!
>He's an obese white trash barbarian with a secret. She's a virginal thirtysomething traffic cop with the power to bend men's minds. They fight crime!
lol
Put a guinea pig in there and you'd get the same effect.
This was really funny :)
http://dailywrong.com/man-finally-comfortable-just-holding-a...
http://dailywrong.com/sample-page/
That said, with a little tweaking, these technologies can - and probably already are - being used to churn out blogspam websites left and right, fully automatic.
Easiest way for free SSL would be to just throw the domain on CloudFlare :)
Now I'm using that content in the video game. I wonder if you could use these articles as some fake news in your game, too. :)
Thank you! Bookmarked!
If you want to see some really creepy AI generated human "photo" faces, take a look at Bots of New York:
https://www.facebook.com/botsofnewyork
How about wording your comment in a way that highlights why it’s a shame these pictures aren’t accessible for those without a Facebook account, and skip the whole “you’re murdering puppies” bit?
The hyperbole is good marketing for a certain audience.
Hot dang. Some Reddit subs can be auto-generated now.
https://www.reddit.com/r/SubSimulatorGPT2/
but yea, it will have generated images now.
I guess Markov chains for the headings, Dall-E for images, maybe GPTx for comments. And/or the GPT models should be made wackier somehow—less coherent, perhaps.
Can DALL-E render Bat Boy?
http://dailywrong.com/wp-content/uploads/2022/07/DALL%C2%B7E...
http://dailywrong.com/wp-content/uploads/2022/07/DALL%C2%B7E...
The images don’t scale properly with the rest of the site, they’re massive compared to the content.
Why do the images load so slow though?
Unless I'm missing something, these seem pretty darn good
Once the API is released, this will be easier to do in a programmatic fashion.
Note: Depending on how many times you do this... I could see there being a continuity problem with the extremes of the image (eg: the far left has no knowledge of the far right). An alternative could be to scale the image down and mask the borders then later scale it back up to the desired resolution.
This scale and mask strategy also works well for images where part of the scene has been clipped that you want to include (EG: Part of a character's body outside the original image dimensions). Scale the image down, then mask the border region, and provide that to the generation step.
https://apps.apple.com/us/app/waifu2x/id1286485858
I paid extra to get the higher quality model using the in-app purchase option. It crushes the phone's battery life, but runs in only ~10 seconds on an iPhone 13 Pro for a single 1000x1000 input image.
Try https://github.com/n00mkrad/cupscale
Considering waifu2x is the name of an algorithm I assumed it was just that algorithm. There's also no mention of other models on the demo page or the Github page as far as I can see.
The confusion originates from the fact that I was using a GUI project for Waifu2x called "Waifu2x Extension GUI" (https://github.com/AaronFeng753/Waifu2x-Extension-GUI) which other than Waifu2x also supports other algorithms like Real-ESRGAN, Real-CUGAN, SRMD, RealSR, Anime4K, RIFE, IFRNet, CAIN, DAIN, and ACNet.
So as you said Cupscale is surely more advanced than Waifu2x (the single algorithm), but do you think it's also better than Waifu2x Extension GUI?
https://www.topazlabs.com/gigapixel-ai
No kidding.
On-demand stock photo generation probably is the next step, particularly when combined with other free media services (Unsplash immediately comes to mind). Simply choose a "look" or base image, add contextual details, and out pops a 1 of 1 stock photo at a fraction of the cost of standard licensing. It'll be very exciting seeing what new products/services will make use of the DALL-E API, how and where they integrate with other APIs, use cases, value adds like upscaling and formatting, etc.
And this is the first picture I got: https://labs.openai.com/s/lSWOnxbHBYQAtli9CYlZGqcZ
It got it a bit strong on the depth of field and I don’t like the angle but I could iterate a few times and get a good one.
For the first few days when it was announced I use to look deep even in real photos in search of generative artifacts. They are not so difficult to spot now, most of the times anyway.
Heck: If the cost to entry is prohibitively low they might do it at a loss and take over the site
It's very good at generating art style images. These kind of images are mostly amazing most of the times. But the Photorealistic images only work with cherry picking.
Me and you must have very different definitions of "cherry picking". For prompts that fall within it's scope (i.e not something unusually complex or obscure) I get usable results probably 90% of the time.
Can you give me some examples of prompts that you tried where you found good results difficult to obtain?
It did generate good dslr like face closeups, as good as Nvidia does, most of the times but not always. Sometimes there are weird artifacts and face does not make sense.
Dslr style blurry photos are mostly good. From the looks of images I follow, imagen is probably more believable. Don't know how much cherry picking goes on there. See this thread [1] for example. I failed to generate image like this (honey dress) in dalle2.
[1]: https://www.reddit.com/r/ImagenAI/comments/w3saku/creating_i...
Maybe it would be cheaper. I imagine it would one day. And maybe it would have a more liberal usage license.
At any rate, I look forward to this. And I look forward to the inevitable debates over which is better: AI generation or photographer.
Turns out that they randomly, silently modify your prompt text to append words like "black male" or "female". See https://twitter.com/jd_pressman/status/1549523790060605440
I don't know which emotion I feel more - applause at how glorious this hack is or tears at how ugly it is.
Good luck to them!
A mortgage AI that calculates premiums for the public shouldn't bias against people with historically black names, for example.
This problem is harder to tackle because it is difficult to expose and resign the "latent space" that results in these biases; it's difficult to massage the ML algo's to identify and remove the pathways that result in this bias.
It's simply much easier to allow the robot to be bias/racist/reflective of "reality" (its training data), and add a filter / band-aid on top; which is what they've attempted.
when this is appropriate is the more cultured question; I don't think we should attempt to band-aid these models, but for more socially-critical things, it is definitely appropriate.
It's naive on either extreme: do we reject reality, and substitute or own? Or do we call our substitute reality, and hope the zeitgeist follows?
That's a great example, thanks. Also, I hope the teams working on that come up with a different solution...
My question to you is: is an algorithm that takes no racial inputs (name, race, address, etc) yet still produces disproportionate results biased or racist? I say no.
[1] https://files.stlouisfed.org/files/htdocs/publications/es/08...
The government, and many people, have moved the definition and goal posts; so that anything that has the end result of a non-proportional uniformity can be labeled and treated as bias.
Ultimately it is a nuanced game; is discriminating against certain clothing or hair-styles racist? Of course. Yet, neither of those are explicitly tied to one's skin color or ethnicity, but are an indirect associative trait because of culture.
In America, we have intentionally muddled the waters of demarcation between culture and race, and are starting to see the cost of that.
Not that I agree with that but I don't see why you would build one otherwise, if you wanted discrimination free mortgages wouldn't the whole process by anonymized and minimal personal information rather than the current system of having to hand over every detail of your life.
1. The training data would've been the best way to get organic results, the input is where it'd be necessary to have representative samples of populations.
2. If the reason the model needs to be manipulated to include more "diversity" is that there wasn't enough "diversity" in the training set then its likely the results will be lower quality
3. People should be free to manipulate the results how they wish, a base model without arbitrary manipulations of "diversity" would be the best starting point to allow users to get the appropriate results
4. A "diverse" group of people depends on a variety of different circumstances, if their method of increasing it is as naive as some of the are claiming this could result in absurdities when generating historical images or images relating to specific locations/cultures where things will be LESS representative
If I want to produce "animal images" but it only produces images of black cats, do you think there is any question whether it's a problem or not?
The "issue" is a different one: that training data - IE, reality, has _unwanted_ biases in it, because reality is biased.
Producing images of men when prompting for "trash collecting workers" should not be much of a surprise: 99% of garbage collection/refuse is handled by men. I doubt most will consider this a "problem," because of one's own bias, nobody cares about women being represented for a "shitty" job.
But ask for picture of CEOs, and then act surprised when most images are of white men? Only outrage, when proportionally, CEO's are, on average, white men.
The "problem" arises when we use these tools to make decisions and further affect society - it has the obvious issue of further entrenching stereotypical associations.
This is not that. Asking DALLE for a bunch of football players, would expectedly produce a huddled group of black men. No issue, because the NFL are disproportionately black men. No outrage, either.
Asking DALLE for a group of criminals, likewise, produces a group of black men. Outage! Except statistically, this is not a surprise, as a disproportionate amount of criminals are black men.
The "problem" is with reality being used as training data. The "problem" is with our reality, not the tooling.
Except in the cases where these toolings are being used to affect society - the obvious example being insurance ML algorithms. et al - we should strive to fix the issues present in reality, not hide them with handicapped training data, and malformed inputs.
I think, for about 95% of the world football is synonymous with soccer. Its kind of interesting that you take this particular example to represent what reality looks like statistically
This is not great. Only about 57% of NFL players are black, and the percentage is more like 47% among college players. It would be better to at least reflect the diversity of the field, even if you don't think it should be widened in the name of dispelling stereotypes.
> Asking DALLE for a group of criminals, likewise, produces a group of black men. Outage! Except statistically, this is not a surprise, as a disproportionate amount of criminals are black men.
Only about 1/3 of US prisoners are black. (Not quite the same as "criminals" but of course we don't always know who is committing crimes, only who is charged or convicted.) That's disproportionate to their population, but it's not even close to a majority. If DALLE were to exclusively or primarily return images of black men for "criminals", then it would be reinforcing a harmful stereotype that does not reflect reality.
"criminals" producing most black people actually would be a perfect example of bias in DALL-E that is arguably racism.
Black people commit a diproportionate amoumt of crime (for a variety of socioeconomic reasons I won't get into here), but even so white people make up a majority of criminals (because white people are the largest ethnic group by far).
Thus, a random group of criminals, if representive of reality, should be majority white.
It's a hard problem for sure. But remember, the bias ends with the user using the tool. If I want a black scientist, I can just say "black scientist".
Let me be mindful of the bias, until we have a generally intelligent system that can actually do it. I'm generally intelligent too, you know.
That is a really, really, narrow viewpoint. I think what people would prefer is that if you query "Scientist" that the images returned are as likely to be any combination of gender and race. It's not that a group is "fragile", it's that they have to specify race and gender at all, when that specificity is not part of the intention. It seems that they recognize that querying "Scientist" will predominantly skew a certain way, and they're trying in some way to unskew.
Or, perhaps, you'd rather that the query be really, really specific? like: "an adult human of any gender and any race and skin color dressed in a laboratory coat...", but I would much rather just say "a scientist" and have the system recognize that anyone can be a scientist.
And then if I need to be specific, then I would be happy to say "a black-haired scientist"
It's way bigger than just this narrow race issue the current zeitgeist is concerned about.
But I agree, maybe I should skew to being optimistic that at least we're trying
I wonder what the distribution of those modifications is?
And the best of all - it does have a meme community around it, and you can always donate if you feel it adds value to your life
I tried the first whimsical, benign thing I could think of: "indiana jones eating spaghetti." The results are clearly recognizable as that. But they are also a kaleidoscope of body horror; a Indiana Jones monster melted into Cthulu forms inhaling plates that are slightly not spaghetti.
I'm interested in using Dall-E commercially, but I think some competitor offering sampling with raw input will have a better chance at my wallet.
I don't understand the relevance of the black box's scrutability - I just want to play with the black box. I am interested in increasing my understanding of the black box, not of a trust-me-it's-great-our-intern-steve-made-it black box derivative.
These kinds of modifications are obviously different. At least the mathematical transformations are attempting at least some level of fidelity to user input, these ones aren't (e.g. someone mentioned they're sometimes getting androgynous results and speculates the added terms are conflicting with the ones they provided in their input). Not all black boxes are equivalent.
This feels like a very hacky way to essentially reinvent programming badly.
My bet is that in a few years or so only a small cohort of engineering and product people will even remember Dall-E and GTP-3 and someone cringe at how we all thought this was going to be a big thing in the space.
There's are both really fascinating novelties, but at the end of the day that's all they are.
[0]: https://dallery.gallery/wp-content/uploads/2022/07/The-DALL%... (PDF)
I hope that every science teacher that can - provide this to every student. This is the future they live in now. They should know these as well as they know how to install an app on a device.
Wait until we have a DALL-E -- Enabled Custom EMOJI stream - whereby, every text you send out has it corresponding DALL-E resultant image for every txt --
Then we can compare images from different people at different times but the prompt was identical... and see what the resultant library of emoji<-->PROMPT looks like?
What about using Dall-e as a watermark for 'nft' signature 'notary' of an email.
If DALL-E provided a unique PID# for every image - and that PID was a key that only the OP runner of the image has - it can be used to authenticate an image to a text source... ??? (Assuming that no two prompts have the same result ever, but assigning a unique id that CAN be used to replay the image to verify it was generated when an original email/SMS was actually sent - it could be a unique way to timestamp authenticity/provenance of a thing...