But can you generate a video just by typing some text?
Can I?
Can we do this yet please?! We should storm Google and violently overthrow them to force them into releasing Dreambooth to everyone.
(I'm probably a little too excited for ML sometimes, but I can't help it. It's the clearest evidence of living in the future, and it's right at our doorstep.)
As a designer, what I’d like more is to be presented with a varied range of generated designs and components for each prompt. I’m more interested in being inspired by a multitude of approaches so that I can design an interface that suits my specific needs.
Perhaps you can present a few options with the ability to select / upvote preferred traits of each, then use that to generate subsequent designs.
You should also capture / generate brand guidelines upfront to ensure designs are consistent.
I'm not experienced in the field of UX, but they're using the word "complex" in such a way that is off-putting to me. I don't want a complex design; I want a simple, intuitive design. And in fact I would think simplicity is the hallmark of good design.
"Simplicity is the ultimate sophistication" - leonardo da vinci
I too am thrown off by this. I also don't see any substance past their video. I wonder if there are any tools out there that actually have working prototypes
After seeing how quickly Midjourney and Stable Diffusion improved, I figured it wasn't long before we'd see AI interface work.
Can this create a similar set of screens based on a single previous prompt? Back in December I experimented illustrating a children's book with Midjourney. The most difficult part was getting a consistent character. Eventually I gave up and had images where his face was obscured, or not showing. There are more constraints in UI design than in character design—so it seems like that might be possible. If so, this would be a lot more interesting.
Yeah, like maybe a sad face instead of a logout button, which doesn't hint at it's functionality until you actually click it. Or a raised eyebrow emoji for resetting your password. A hand with six fingers to modify accessibility features...
Generate repetitive, default not interesting UIs and get stuck in prompt hell where being specific takes more time than actual designing and implementing with UI framework.
Ai revolution looks more and more like WYSIWYG editors hype long time ago.
Every manager thinks that he will be able to do everything on their own with AI but it will end with creating jobs like Senior AI Prompt Developer...
Reminds me of all that hype around Business Process Management (BPM) and we ended up with "Integration Specialists" to actually build the process flows because the fancy diagram builders only went so far.
This is useful for probably the lower 20% difficulty of UI work. And I bet that developers will get (over)loaded with the responsibility of writing prompts instead of a new job being created.
While I tend to agree, I’m also somewhat hopeful it can be a force multiplier for myself. I don’t think it will actually replace designers, anymore than WYSIWYG editors/Squarespace/Wix et al did. But it might make some design processes quicker for those who are already skilled, and that would be pretty neat.
I don't care if it's powered by AI or gerbils. I care how good the output is.
> curated AI-generated illustrations and images to match your vision and style.
The example is quite ugly
> “I am blown away by the ingenuity and complexity of this machine-learning implementation, the level of fidelity of this engineering challenge is remarkable.”
Aditya Agarwal // Ex-CTO, Dropbox
That's just embarrassing. Did ChatGPT write that quote?
It’s not generating images. This looks like an LLM like GPT-3.
It’s (probably) generating text in some encoding that represents a view hierarchy along with basic layout attributes (alignment etc)
Probably fine-tuned some on human-created examples with the same format.
So the prompt provides the example format and then merges in the user’s request. LLM spits output that is then parsed and transform into css/figma etc.
That’s almost surely how it is done. It is how we do it and it works well. We are just unsure it needs all that complexity; it seems generating ‘educated guesses’ works much better. Probably a combination will be a winner.
UI generation is an interesting problem. It's very from other domains like copywriting, search, etc. that these new AI models excel at. For it to be useful in the UI field, there are quite a few things more that are needed. Something like this is just one step above using stable diffusion for generating images of prototypes to get good color schemes, and applying that to a template off of website marketplace like envato.
One thing that makes ui different is that it's not that difficult to adapt a design for multiple purposes. Often times just changing the images. That's very different from images where you can't just swap out sections of an image because things like the lighting will be off. It's makes customization very easy, so the tolerance for getting a design and tearing it apart to be able to customize it is a lot lower. You can mockup a presentable design for a button in like a minute, so it's not worth the hassle of using prompts to slowly make adjustments as opposed to images where it's going to take hours to design what a single prompt can do.
Also the key thing with ui design isn't the design - again ui is highly adaptable. You can grab two templates and have it used within the company. You can't grab two different images with different styles and put them in a picture book. The key thing with ui/ux is about communication and making sure everyone is happy and various issues are addressed. It's what made figma so great, because you didn't have to ask people do download photoshop to view a file or if they wanted to tweak something slightly. It's about making sure that small details like the spacing are consistent, and when they can be broken and communicating that with the dev team. It's a bunch of ideas that need to be communicated that happens as the design is developed. If those things don't get produced, that's a problem.
Aside from all the poo-pooing, its huge that you can edit the output in Figma. That will make it very easy to tweak, which is a crucial step to AI generated stuff.
Hi,
1) This is awesome! Would love to give it a try!
2) I actually just released a AI tool that you can start playing today on figma called magicbrushai
With the recent trend of designers submitting their meticulously labeled designs to Figma, there is anticipation that Adobe will soon make a game-changing announcement - the elimination of the need for a separate UI designer.
This will allow everyone to take on the role of designer, thus bringing to light the fundamental truth that design is primarily about solving user problems, followed by making it accessible and aesthetically pleasing experience.
However, the last ten years of UI design, dominated by the influence of Dribble, has led many to believe that they are designers simply because they have acquired basic skills and experimented with variations.
I hope, this shift will give the power to designers, finally, to decouple their minds from the visual bias and personal preferences and connect with the user problems in a meaningful and productive way.
I tend to agree. It makes me think that the roles of PM, UX and Eng will start to converge slowly as AI gets better in each field. People will still need to be product thinkers and storytellers though
If I use all the successful ideas for past products as input for my AI model and then use a series of AI models to create and validate prototypes, why would I need to bring in a human to the loop and risk disrupting the process?
> With the recent trend of designers submitting their meticulously labeled designs to Figma
Oh god. I must have had this thought many times, but the way you structured this sentence finally made it click loud and clear for me: the AI revolution feeds on the previous era of user-generated content.
We've all spent the last ~two decades "submitting our meticulously labeled" content to all kinds of platforms. While it quickly became an open secret that those platforms are taking on the hosting and delivery costs in exchange for exposing us to ads (and profiling us to get a better deal on those ads), I'm not sure if anyone predicted that the user-generated content itself will become pure gold. Right now, every platform that owns a large amount of user-generated content is sitting on a mother lode: a corpus of labeled examples of a type of content, ready to be fed into ML models for training.
Random examples off the top of my head: Facebook could train a ChatGPT on its own corpus of posts and comments, but could also make a model that generates plausibly looking digital records of populations, complete with personality quirks, relationships, photos, etc. Hell, if they get their VR fairy land off the ground, they could actually populate it with such virtual people - solving the problem of NPCs in games being obviously scripted and boring (and also solving the "chicken and egg" problem of a social network by using AI as ultimate astroturfing/sockpuppeting tool).
Makes me wonder if we'll see more and more barriers popping up on those platforms, to make it difficult for competition to just scrap the content at scale.
There have of course been hundreds of threads in this space in recent months—enough that "AI fatigue" threads are now on autorepeat in their own right. Soon we will have AI fatigue fatigue.
In the context of the current AI flurry, I don't think landing pages and waiting lists can count as SNI. Let's keep the space for projects and products that people can actually try (and/or articles that contain genuinely interesting new information).
Funny thing is, at this point you could use those AI models to generate ideas for what to do with AI models (possibly complete with landing pages of imaginary companies doing it), and it will be hard to tell apart from the "real thing". Never before did ideas mean so little, and execution so much.
But people already felt for the fake it till you make it. This will make it even easier, but how many people have a vague idea in a coffee place, register a domain and put a landing page and that’s it? Some put a saas starter on to catch signups and leave the dashboard ‘coming soon’, also forever…
Maybe this will finally kill the whole approach - when everyone and their dog runs fake landing pages for their shower ideas, people might finally wisen up and stop believing them (and giving them their e-mail addresses).
Maybe the founders are just trying to throttle for performance and ability to let in users only as fast as they can talk to them and get feedback. That’s what I feel like Y Combinator taught me anyway. Feels like something we should encourage on HN.
I get the concern about vapor ware and fatigue but feels like there is a better happy medium here.
Everyday we're getting closer to a toolkit that allows a novice to go from an idea to a working product with little more than natural language in-between. It's still a ways off, but the trajectory is clear.
I have a working theory -- which I'm sure I didn't discover first -- that the broader the gap between the traits required for the process of creating a thing, and the traits which would make someone want to use the thing, the worse the design/product outcomes. It applies readily to a field like architecture, where a meticulous, granular attention to geometry and streamlined efficiency aids in the realization of a building plan but rarely coheres with the emotive, intuitive experience of a place by the people who use it. And I think it even better applies to the development of consumer software. To be a good dev you need, among other things, a high degree of patience, while the average user is defined by little else than their near nonexistent attention span. The best practitioners, like Nikita Bier, have fully internalized this -- "consider every tap by a user to be a miracle"[0]. But it's an easier lesson to rote memorize than to put into practice. Other incongruences between the process of creating and using abound: organization vs complexity, optionality vs usability, minimalism vs ornamentation, etc. They all create friction for the best version of an idea coming to fruition.
To borrow from the analogy of architecture again, we may be entering the era of vernacular software design -- the meta-discipline of development as such begins to fade away, and what counts is one's ability to find, articulate, and solve a problem, or to create a novel experience. Like their real-world architectural equivalents, the resulting designs will probably be maligned by the field for esoteric reasons, or regarded as unrefined and inferior. But also like real-world vernacular architecture, they'll probably be widely enjoyed by the people who actually use them. I'm excited for that future, and this service -- if it works as well as is showcased here -- is a big part of getting there.
I'm not disagreeing, but I'm also not fully convinced. My working theory is that it's less about the gap between traits of creators vs. users, but more about the users' ability to adapt and iterate on the thing they're using.
In architecture case, people living in a building can't do all that much to adapt it or iterate on its design - but, together with their neighbors, they have much greater influence on the shape of the town or district they live in. As things are today, you can't tell architects to let the future tenants weigh in on the building plan and expect the building to actually be built - but you can get hands-off with the urban planning and let it happen organically (and AFAIK the common wisdom these days is that this is the right thing to do).
Thing is, I feel the reason for the building vs. district difference has less to do with traits, and more with the substrate. Constructing a building is expensive in terms of labor, material and expertise, and is best done in a focused manner. There's hardly anything that's malleable here at our current technology level. Conversely, the buildings are only a part of what makes a livable area, and everything else - what kind of businesses and facilities open, where they open, which space is left for verdure and which gets paved - is something that happens over time, slowly, in a distributed and feedback-controlled way.
Moving on to software: as things are right now, software is mostly built like buildings, with an extra kick that the developers are even more exploitative and abusive of their tenants. Not only software is not malleable, it's usually in an adversarial relationship with its users.
Would adding natural language to the mix help? I'm not sure. Would giving everyone a 3D printer that can print you a multi-story building from a pile of SketchUp drawings help make buildings more coherent with "the emotive, intuitive experience of a place by the people who use it"? To some extent, maybe, but I feel the limiting factor here isn't the building or the software - it's the district and the software ecosystem.
As any one of us who know how to code can attest, there's only so much you can actually do with code in practice: most of the useful and interesting things happen in the networks - the relationship graphs that, through computer systems, connect people who want something with people who can provide it - be it data, or product, or service, or experience, or impact on the physical reality. So the real limiting factor is society - the economy, the laws and the culture that constraint what any one of us can do. And before that, a software ecosystem that's increasingly locked down and hard to work with.
(All that, plus I also think that programming via natural language makes as much sense as constructing buildings from napkin sketches: it works - but only when you're providing your input to a professional programmer/architect. To make it work without a human intermediary, you'd have to create AI programmers / AI architects - this, and not natural language understanding, is the limiting factor.)
All the images on the site show mobile apps, does this also do website app design?
Also, I'd love to see how it handles design consistency between pages. Can this be used to come up with a design and component system, that is used on multiple pages (eg. come up with the components, then each page is a layout of those saved components)?
57 comments
[ 6.9 ms ] story [ 135 ms ] threadCan I?
Can we do this yet please?! We should storm Google and violently overthrow them to force them into releasing Dreambooth to everyone.
(I'm probably a little too excited for ML sometimes, but I can't help it. It's the clearest evidence of living in the future, and it's right at our doorstep.)
As a designer, what I’d like more is to be presented with a varied range of generated designs and components for each prompt. I’m more interested in being inspired by a multitude of approaches so that I can design an interface that suits my specific needs.
Perhaps you can present a few options with the ability to select / upvote preferred traits of each, then use that to generate subsequent designs.
You should also capture / generate brand guidelines upfront to ensure designs are consistent.
https://en.wikipedia.org/wiki/Design_language
I too am thrown off by this. I also don't see any substance past their video. I wonder if there are any tools out there that actually have working prototypes
Can this create a similar set of screens based on a single previous prompt? Back in December I experimented illustrating a children's book with Midjourney. The most difficult part was getting a consistent character. Eventually I gave up and had images where his face was obscured, or not showing. There are more constraints in UI design than in character design—so it seems like that might be possible. If so, this would be a lot more interesting.
EDIT: Link if anyone is interested.
https://solomon.io/childrens-story-written-illustrated-ai/
Ai revolution looks more and more like WYSIWYG editors hype long time ago. Every manager thinks that he will be able to do everything on their own with AI but it will end with creating jobs like Senior AI Prompt Developer...
I suggest this might be fun for ideatio though.
I dislike this.
I don't care if it's powered by AI or gerbils. I care how good the output is.
> curated AI-generated illustrations and images to match your vision and style.
The example is quite ugly
> “I am blown away by the ingenuity and complexity of this machine-learning implementation, the level of fidelity of this engineering challenge is remarkable.” Aditya Agarwal // Ex-CTO, Dropbox
That's just embarrassing. Did ChatGPT write that quote?
What does behind the scenes might isn’t necessarily 1:1 with old image generation stuff either way.
It’s (probably) generating text in some encoding that represents a view hierarchy along with basic layout attributes (alignment etc)
Probably fine-tuned some on human-created examples with the same format.
So the prompt provides the example format and then merges in the user’s request. LLM spits output that is then parsed and transform into css/figma etc.
UI generation is an interesting problem. It's very from other domains like copywriting, search, etc. that these new AI models excel at. For it to be useful in the UI field, there are quite a few things more that are needed. Something like this is just one step above using stable diffusion for generating images of prototypes to get good color schemes, and applying that to a template off of website marketplace like envato.
One thing that makes ui different is that it's not that difficult to adapt a design for multiple purposes. Often times just changing the images. That's very different from images where you can't just swap out sections of an image because things like the lighting will be off. It's makes customization very easy, so the tolerance for getting a design and tearing it apart to be able to customize it is a lot lower. You can mockup a presentable design for a button in like a minute, so it's not worth the hassle of using prompts to slowly make adjustments as opposed to images where it's going to take hours to design what a single prompt can do.
Also the key thing with ui design isn't the design - again ui is highly adaptable. You can grab two templates and have it used within the company. You can't grab two different images with different styles and put them in a picture book. The key thing with ui/ux is about communication and making sure everyone is happy and various issues are addressed. It's what made figma so great, because you didn't have to ask people do download photoshop to view a file or if they wanted to tweak something slightly. It's about making sure that small details like the spacing are consistent, and when they can be broken and communicating that with the dev team. It's a bunch of ideas that need to be communicated that happens as the design is developed. If those things don't get produced, that's a problem.
Check it out: https://www.magicbrushai.com/
Would love some feedback .. and congrats to the Galileo team for launching!
This will allow everyone to take on the role of designer, thus bringing to light the fundamental truth that design is primarily about solving user problems, followed by making it accessible and aesthetically pleasing experience.
However, the last ten years of UI design, dominated by the influence of Dribble, has led many to believe that they are designers simply because they have acquired basic skills and experimented with variations.
I hope, this shift will give the power to designers, finally, to decouple their minds from the visual bias and personal preferences and connect with the user problems in a meaningful and productive way.
Oh god. I must have had this thought many times, but the way you structured this sentence finally made it click loud and clear for me: the AI revolution feeds on the previous era of user-generated content.
We've all spent the last ~two decades "submitting our meticulously labeled" content to all kinds of platforms. While it quickly became an open secret that those platforms are taking on the hosting and delivery costs in exchange for exposing us to ads (and profiling us to get a better deal on those ads), I'm not sure if anyone predicted that the user-generated content itself will become pure gold. Right now, every platform that owns a large amount of user-generated content is sitting on a mother lode: a corpus of labeled examples of a type of content, ready to be fed into ML models for training.
Random examples off the top of my head: Facebook could train a ChatGPT on its own corpus of posts and comments, but could also make a model that generates plausibly looking digital records of populations, complete with personality quirks, relationships, photos, etc. Hell, if they get their VR fairy land off the ground, they could actually populate it with such virtual people - solving the problem of NPCs in games being obviously scripted and boring (and also solving the "chicken and egg" problem of a social network by using AI as ultimate astroturfing/sockpuppeting tool).
Makes me wonder if we'll see more and more barriers popping up on those platforms, to make it difficult for competition to just scrap the content at scale.
https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
Edit: something similar came up yesterday: https://news.ycombinator.com/item?id=34735331.
There have of course been hundreds of threads in this space in recent months—enough that "AI fatigue" threads are now on autorepeat in their own right. Soon we will have AI fatigue fatigue.
When there's a major ongoing topic (MOT) like this, our approach is to downweight the follow-ups and try to clear space for the submissions that have significant new information (SNI): https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
In the context of the current AI flurry, I don't think landing pages and waiting lists can count as SNI. Let's keep the space for projects and products that people can actually try (and/or articles that contain genuinely interesting new information).
I get the concern about vapor ware and fatigue but feels like there is a better happy medium here.
I have a working theory -- which I'm sure I didn't discover first -- that the broader the gap between the traits required for the process of creating a thing, and the traits which would make someone want to use the thing, the worse the design/product outcomes. It applies readily to a field like architecture, where a meticulous, granular attention to geometry and streamlined efficiency aids in the realization of a building plan but rarely coheres with the emotive, intuitive experience of a place by the people who use it. And I think it even better applies to the development of consumer software. To be a good dev you need, among other things, a high degree of patience, while the average user is defined by little else than their near nonexistent attention span. The best practitioners, like Nikita Bier, have fully internalized this -- "consider every tap by a user to be a miracle"[0]. But it's an easier lesson to rote memorize than to put into practice. Other incongruences between the process of creating and using abound: organization vs complexity, optionality vs usability, minimalism vs ornamentation, etc. They all create friction for the best version of an idea coming to fruition.
To borrow from the analogy of architecture again, we may be entering the era of vernacular software design -- the meta-discipline of development as such begins to fade away, and what counts is one's ability to find, articulate, and solve a problem, or to create a novel experience. Like their real-world architectural equivalents, the resulting designs will probably be maligned by the field for esoteric reasons, or regarded as unrefined and inferior. But also like real-world vernacular architecture, they'll probably be widely enjoyed by the people who actually use them. I'm excited for that future, and this service -- if it works as well as is showcased here -- is a big part of getting there.
[0] https://twitter.com/nikitabier/status/1608310890243444738
In architecture case, people living in a building can't do all that much to adapt it or iterate on its design - but, together with their neighbors, they have much greater influence on the shape of the town or district they live in. As things are today, you can't tell architects to let the future tenants weigh in on the building plan and expect the building to actually be built - but you can get hands-off with the urban planning and let it happen organically (and AFAIK the common wisdom these days is that this is the right thing to do).
Thing is, I feel the reason for the building vs. district difference has less to do with traits, and more with the substrate. Constructing a building is expensive in terms of labor, material and expertise, and is best done in a focused manner. There's hardly anything that's malleable here at our current technology level. Conversely, the buildings are only a part of what makes a livable area, and everything else - what kind of businesses and facilities open, where they open, which space is left for verdure and which gets paved - is something that happens over time, slowly, in a distributed and feedback-controlled way.
Moving on to software: as things are right now, software is mostly built like buildings, with an extra kick that the developers are even more exploitative and abusive of their tenants. Not only software is not malleable, it's usually in an adversarial relationship with its users.
Would adding natural language to the mix help? I'm not sure. Would giving everyone a 3D printer that can print you a multi-story building from a pile of SketchUp drawings help make buildings more coherent with "the emotive, intuitive experience of a place by the people who use it"? To some extent, maybe, but I feel the limiting factor here isn't the building or the software - it's the district and the software ecosystem.
As any one of us who know how to code can attest, there's only so much you can actually do with code in practice: most of the useful and interesting things happen in the networks - the relationship graphs that, through computer systems, connect people who want something with people who can provide it - be it data, or product, or service, or experience, or impact on the physical reality. So the real limiting factor is society - the economy, the laws and the culture that constraint what any one of us can do. And before that, a software ecosystem that's increasingly locked down and hard to work with.
(All that, plus I also think that programming via natural language makes as much sense as constructing buildings from napkin sketches: it works - but only when you're providing your input to a professional programmer/architect. To make it work without a human intermediary, you'd have to create AI programmers / AI architects - this, and not natural language understanding, is the limiting factor.)
All the images on the site show mobile apps, does this also do website app design?
Also, I'd love to see how it handles design consistency between pages. Can this be used to come up with a design and component system, that is used on multiple pages (eg. come up with the components, then each page is a layout of those saved components)?