The samples included in the press release are quite impressive to my ears, but the other samples (especially from AudioGen) have a hint of artificially.
As usual the music is quite repetitive, but I'm looking forward to tools that simplify changing the prompt whilst it generates over a window. I can only imagine the consequences for royalty free music.
Edit: the "Text-to-music generation with diffusion-based EnCodec" samples are quite impressive.
Meta has several of the biggest UGC platforms, and in this case the complement is content itself. Reels with autogenerated (and royalty free) background music is the obvious example but I'm sure there are more. Maybe creative for ads as well?
In short term it's social media, because people will share whatever they generate on social media. But I don't think it's a very strong incentive to invest in AI for Meta.
To Metaverse access. Filling the metaverse with engaging interactive 3D content is an insane job with 2020 technology. It requires a huge amount of a range of skilled labor to create 3D models, soundtracks, NPC dialog, visuals et.c. to make a compelling experience. In 2030 that may have been reduced to that everyone with creativity and Internet access can do it. Sure, most of it will be silly things - but so is social media today, does not make it any less of a commercial success. And there will be be millions of semi-pro creators to create the things with higher production value, like with videography today.
They want to comoditize the offerings of OpenAI, Google, MS, and Apple. Also, they gain mindshare and good will after years of bad publicity. Some back contributions might help them improve the models for free.
If they just keep their models, people won't be interested and will build over ChatGPT or Bard.
The same move that Microsoft did back in 90's to kill Netscape. Make your product the one available to masses, next generation of users will be using your product.
I just can't get how bad Google is doing. They have a ton of top researchers, papers, money, just no good LLMs. It's like OpenAI was first to the punch, and everyone else just saw $$$. Meta was smart to go down this open source road, as the masses will start training their llamas one way or another.
Personally I believe the "intelligence" aspect will asymptote, so even having exclusive access to a "super AI" (i.e. hypothetical 1T parameter model like a GPT5) won't be that much of a step behind the lesser AIs, and as soon as you grant access to the masses they will start to use some transfer learning to make their "lesser" models better.
AI applications though still need a lot of work. The models are smart or general purpose enough to be useful to the average person out of the box.
The problem also is that Google is making lot of grandiose announcements about tools and models that nobody can see nor use. This is a serious credibility problem in the long-term.
You can use some of them. They have an “AI powered” search (as if their previous search isn’t considered AI anymore). It’s an experiment you can turn on. For programming questions it’s not terrible.
That said, there are a ton of “look at this cool thing out research team did” and then you never hear about them again things from Google. They even built a music generator that was closed to the public until recently.
The fact you believe, rightly or wrongly, that meta is ahead of google on ai explains why meta would open source this. It’s a good reputation to maintain.
A competitive opensource project basically destroys the pricing power of all closed-source alternatives.
If you're a company and wanted to integrate an LLM into your product if the choice is between several equally good models, but one is free and open-source which would you pick?
Aside from keeping competition at bay, this move also gives Meta leverage because ecosystems are now being built around their projects. If these models see wide-scale adoption they could later launch AudioCraft+ as a licensed version with some extra features for example.
Alternatively, they might offer support or hosting for their open source projects.
Right now though I think the primary benefit of these open sourced models is to attract talent. If Meta is seen as one of the leaders in AI then researchers will want to work for them simply for the prestige.
Arguably one of the reasons Meta has been behind so many awesome projects like PyTorch and React over the last decade was because they were seen as the cool place for recently graduated, but talented software engineers to work in ~2010.
Was asking myself the same earlier, I'm sure it is largely to do with publicity and the fact that selling these services is not their core business. At the very least releasing this stuff probably won't damage their core business but will take the sheen off of some other big names.
I wondered though, generative AI is hurling us into a world where we'll need more mechanisms to sort real from fake, provenance will play a large part, and meta's platforms could be part of the answer. i.e. content linked to actual verifiable people.
Quite impressive although if these are the cherry picked examples the average output must be pretty weak! Nothing catchy about most of these examples and the reggae one is pretty lame.
Something like the algorithm TikTok uses. First probing by offering a variety of content that should match based on what little information you have on the user (ip location, locale, etc).
Then use the user’s action to iteratively refine your classification, until you end up with something tailor-made.
They should start with AudioCraft itself, conceptually it's derivative work and it doesn't matter if it's "open source" or not. Try throwing in someone's sample in a song and publish it saying "no copyright infringement intended and I totally don't make any money from it"... If it becomes popular, see how long it stays up until DMCA takedown. And we know this dataset is already popular.
Even before streaming, You never "owned" any music legally [1], you merely owned a physical copy of a performance[3] of a song, that in no way gives you the right to make derivative works [2] automatically.
Also it doesn't really matter on what the law says, RIAA in the last iteration, relied on the fact it you would rather pay a fine than be able to afford expensive lawyers to fight the specifics out in court on average.
It was always about disproportionate ability to bring resources against individual "offenders" to create fear among everyone to deter "undesirable" forms of copying, not necessarily what the legal protections were.
---
[1] Unless you specifically commissioned it under a contract which gave you the right
[2] See recent cases including those related to Kris Kashtanova and Andy Warhol.
Establishing derivation is at the crux of all legal matters surrounding diffusion models. It has not yet been clearly established. If it is, then I'd agree with you. Until then, I think it's a bit more up in the air.
Also, IIRC, RIAA did not bring many resources to bear against e.g. "home taping" itself, because they could essentially never know that it had occured. The overwhelming majority of their efforts went into trying to takedown people distributing multiple copies.
The Kashtanova case does not cover derivation in any real way, but is really about copyright attribution choices between human and software.
The Warhol case specifically tests a fair use claim, not a derivation claim.
They may have gotten the AHRA passed, but they essentially lost in every important way:
> "This exception was crucial in RIAA v. Diamond Multimedia Systems, Inc.,[14] the only case in which the AHRA's provisions have been examined by the federal courts. The RIAA filed suit to enjoin the manufacture and distribution of the Rio PMP300, one of the first portable MP3 players, because it did not include the SCMS copy protection required by the act, and Diamond did not intend to pay royalties. The 9th Circuit, affirming the earlier District Court ruling in favor of Diamond Multimedia,[15] ruled that the "digital music recording" for the purposes of the act was not intended to include songs fixed on computer hard drives. The court also held that the Rio was not a digital audio recording device for the purposes of the AHRA, because 1) the Rio reproduced files from computer hard drives, which were specifically exempted from the SCMS and Royalty payments under the act, 2) could not directly record from the radio or other transmissions. "
From the AHRA itself:
> No action may be brought under this title alleging infringement of copyright based on the manufacture, importation, or distribution of a digital audio recording device, a digital audio recording medium, an analog recording device, or an analog recording medium, or based on the noncommercial use by a consumer of such a device or medium for making digital musical recordings or analog musical recordings.
and from Wikipedia again:
> In regard to home taping, the provision broadly permits noncommercial, private recording to analog devices and media. However, it fails to resolve the home taping debate "conclusively," as it only permits noncommercial, private recording to digital devices and media when certain technology is used.
> Two reports by the House of Representatives characterize the provision as legalizing digital home copying to the same degree as analog. One states "in the case of home taping, the exemption protects all noncommercial copying by consumers of digital and analog recordings,"[22] and the other states "In short, the reported legislation [Section 1008] would clearly establish that consumers cannot be sued for making analog or digital audio copies for private noncommercial use."[23]
Similarly, language in the RIAA v. Diamond Multimedia decision suggests a broader reading of the Section 1008 exemptions, providing blanket protection for "all noncommercial copying by consumers of digital and analog musical recordings" and equating the spaceshifting of audio with the fair use protections afforded home video recordings in Sony v. Universal Studios:
>> In fact, the Rio's operation is entirely consistent with the Act's main purpose – the facilitation of personal use.
RIAA and others in general sued a lot of people including bar owners for playing their songs etc , there were also some enforcement via private companies with three strike policies and so on especially in Europe .
They went after ISPs and torrent sites which only hosted magnet links and many others who shouldn’t have been really sued
The goal was to create a very hostile environment for downloading songs to protect their interests - “you wouldn’t download a car!”
It was never the goal nor ever realistic to actually pursue enforcement action against every offender , the idea was to change behavior with all the related actions .
They did end up changing behavior, people just didn’t want the hassle, or be in fear so paying for streaming for access had a stronger value proposition, it was not what the RIAA planned , but benefits enormously today.
> that in no way gives you the right to make derivative works
True, but only because you have that right anyway. I can do anything I like with copyrighted content I legally possess, as long as I don't distribute the results of my efforts.
Legal acquisition does not matter for AI training. If training is fair use then you can train on pirated material (e.g. OpenAI GPT). If it's not fair use then buying the material does not matter, you have to negotiate a specific license for AI training for each work in the training set, which is impractical at the scales most AI companies want to work.
If you purchase the music, you have a (sometimes explicit, sometimes implicit) license to do certain things with the music, entirely independent of any concept of "fair use".
The question is not "is training part of fair use?" but "is training part of, implicitly or explicitly, the rights I already have after purchase?"
Given that "training" can be done by simple playing the music in the presence of a computer with its microphone turned on, it's not clear how this plays out legally.
In the US, exceptions to copyright come across in two distinct bundles: first sale and fair use. They exist specifically because of the intersection between copyright law and two other principles of the US constitution:
- First sale: The Takings Clause prohibits government theft of private property without compensation. Because copyright owners are using a government-granted monopoly to enforce their rights, we have to bound those rights to avoid copyright owners being able to just come and take copies of books or music you've lawfully purchased.
- Fair use: The 1st Amendment prohibits government prohibitions on free speech. Because copyright owners are using a government-granted monopoly to enforce their rights, we have to bound those rights to avoid copyright owners being able to censor you.
If you hinge your argument on "I bought a copy", you're making a first sale argument.
Notably, first sale is limited to acts that do not create copies. This limit was established by the ReDigi case[0]. Copyright doesn't care about the total number of copies in circulation, it cares about the right to create more. So an AI training defense based on first sale grounds would fail because training unequivocally creates copies.
Fair use, on the contrary, does not care if you bought a copy of a work legally. It only cares about balancing your right to speech against the owners' right to a monopoly over theirs. And it has so far been far more resistant to creative industry attempts to limit exceptions to copyright - to the point where I would argue that "fair use" is an effective shorthand for any exception to copyright, including ones in countries that have no fair use doctrine and do not respect judicial precedent.
The courts won't care how the training comes about, just if the act of training an AI alone[1] would compete with licensing the images used in the training set data.
Is training a "pirate model" something you'd reasonably be able to do at home though, given the compute requirements? The analogous "image generation at home" is only possible due to a for-profit entity with significant resources choosing to (a) play fast-and-loose with the provenance of their training set and (b) giving away the resulting model for free, if the open source community had to train their models from scratch then as best as I can tell they would still be stuck in the dark ages generating vague goopy abominations.
Currently, yes, available compute power @ home does indeed seem like a limitation. Whether that remains true going forward seems a little unclear to me.
Perhaps LoRA (Low-Rank Adaptation) training techniques could be used for these types of models, like they're currently being used with LLMs and latent text-to-image diffusion models.
Sadly looks unlikely if the base model wasn't trained on vocals.
> Mitigations: Vocals have been removed from the data source using corresponding tags, and then using a state-of-the-art music source separation method, namely using the open source Hybrid Transformer for Music Source Separation (HT-Demucs).
> Limitations: The model is not able to generate realistic vocals.
Ugh, I dread having to listen to everyone's hyper personal music because they swear up and down to the point of tears that "IT'S THE BEST SONG EVER CREATED! EVER!!!", while the constantly prod for you to affirm how amazing the song is.
Bruh, music is subjective as hell, and I can already tell I hate this song.
> MusicGen, which was trained with Meta-owned and specifically licensed music, generates music from text-based user inputs, while AudioGen, which was trained on public sound effects, generates audio from text-based user inputs.
Meta is really clearly trying to differentiate themselves from OpenAI here. Open source + driving home "we don't use data we haven't paid for / don't own".
This is purely a function of everyone remembering the RIAA's decade-long campaign to prevent people from taking the music they had rightfully stolen. As far as I'm aware LLaMA was trained on "publicly available data"[0], not "licensed data".
Furthermore, MusicGen's weights are licensed CC-BY-NC, which is effectively a nonlicense as there is no noncommercial use you could make of an art generator[1]. This is not only a 'weights-available' license, but it's significantly more restrictive than the morality clause bearing OpenRAIL license that Stability likes to use[2].
Yes, but you can easily make noncommercial use of an art generator.
Obviously, you can't host a commercial art generation service with a noncommercial-use license, and (insofar as art produced by a generator is a derivative work of the model weights, which is a controversial and untested legal theory) you can’t make commercial art with a noncommercial license, but not all art is commercial.
"Noncommercial art" is not a thing in the eyes of the law. Even if you don't intend to make money the law still considers the work itself to be commercial. That's why CC-BY-NC has to have a special "filesharing is non-commercial" statement in it, because people have made successful legal arguments that it is.
You're probably thinking of "not charging a fee to use", which is a subset of all the ways you can monetize a creative work. You can still make money off of AudioCraft by just hosting it with banner ads next to the output. Even a "no monetization" clause[0] would be less onerous than "noncommercial use only", because it'd at least be legal to use AudioCraft for things like background music in offices.
[0] Which already precludes the use of AudioCraft music on YouTube since you can't do unmonetized uploads anymore
> “Noncommercial art” is not a thing in the eyes of the law
The definition of “NonCommercial”, the oddly capitalized term of art in the license, is not a matter of general law, it is a matter of the license, which defines it as “not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange.”
> Even if you don’t intend to make money the law still considers the work itself to be commercial.
Even if you do make money, if the use is “not primarily intended” for that purpose, it is "NonCommercial" in the terms of the license.
> That’s why CC-BY-NC has to have a special “filesharing is non-commercial” statement in it, because people have made successful legal arguments that it is.
It has the filesharing term in it because it permits that particular exchange-of-value as a primary purpose.
> Even a “no monetization” clause would be less onerous than "noncommercial use only"
How would a clause that prohibits monetization entirely be less onerous than one which prohibits it only as the primary intent of use?
> it’d at least be legal to use AudioCraft for things like background music in offices.
It is legal to use it for that purpose (in a for-profit enterprise, I suppose, one might make an argument that any activity was ultimately primarily directed at “commercial advantage”, but in a government or many nonprofit environments, that wouldn’t be the case.)
In their example audio clips they have a "perfect for the beach" audio track. With your understanding of the NC license, would a resort or private beach club be able to play a similar generated music track at their poolside bar or something along those lines? Their primary intention of the bar isn't to play the music, its just an additional ambiance thing; they're trying to sell drinks and have guests pay membership fees, people aren't really coming because of the background music.
> With your understanding of the NC license, would a resort or private beach club be able to play a similar generated music track at their poolside bar or something along those lines?
A resort, probably not, ambiance is, at least arguably, a marketable commercial advantage; a private club in the “mutual benefit organization” sense (rather than a “business selling memberships”, which is just like a resort), probably, because their interest, even indirectly, isn’t making money.
In copyright law the use of the work itself is considered a commercial benefit, so "noncommercial use" is an oxymoron. Consider these situations:
- If I use AudioCraft to post freely-downloadable tracks on my SoundCloud, I still get the benefit of having a large audio catalog in my name, even if I'm not selling the individual tracks. I could later compose tracks on my own and ride off the exposure I got from posting "noncommercially".
- If I run AudioCraft as a background music generator in my store, I save money by not having to license music for public performance.
- If I host AudioCraft on a website and put ads on it, I'm making money by making the work available, even though I'm not charging a fee for entry.
I suspect that a lot of people reading this are going to have different arguments for each. My point is that if you don't think that all of these situations are equally infringing of CC-BY-NC, then you need to explain why some are commercial and some are not. Keep in mind that every exception you make can be easily exploited to strip the NC clause off of the license.
If you're angry at the logic on display here, keep in mind that this is how judges will construe the license, and probably also how Facebook will if you find a way to make any use of their AI. The only thing that stops them from rugpulling you later is explicit guidance in CC-BY-NC. Unfortunately, the only such guidance is that they don't consider P2P filesharing to be a commercial use.
So, absent any other clarifications from Facebook, all you can do without risking a lawsuit is share the weights on BitTorrent.
EDIT: And yes, I have made stuff just to make stuff. I license all of that under copyleft licenses because they express the underlying idea of 'noncommercial' better than actual noncommercial clauses do.
Do you think that non commercial use simply doesn't exist or something?
Because non commercial use isn't some crazy concept. It is a well established one, that doesnt disclude literally everything.
Also, you are ignoring the idea that Facebook will almost certainly not sue anyone for using this for any reason, except possibly Google or Apple.
So if you aren't literally one of those companies you could probably just use it anyway, ignore the license completely, and have zero risk of being sued.
The issue with “non commercial” is that no, it’s not well established. Licenses with a NC clause are so problematic to be practically useless. If you just want to use something at home privately you don’t need a CC license… a CC license is for use and redistribution.
I miss that blog. It was a little crazy and the comments were a flame war shitshow, but man it was fun to read sometimes. Even if I vehemently disagreed, it got me thinking.
Whatever happened to esr? Did he just get too paranoid and clam up?
Funnily enough in the UK they actually tried to fix this. The music industry argued that the lack of a private copying levy made legalized CD ripping into government confiscation of copyright ownership... somehow. The UK courts bought this, so now the UK government is constitutionally mandated to ban CD ripping, which is absolutely stupid.
I knew CD ripping got reversed but not the arguments against it, definitely stupid as not giving a monopoly is not the same as confiscation (seems like a very straightforward reasoning). No doubt done Tory got a 'management consultancy' gig with the RIAA from that one.
I like that it makes software like iTunes contributory infringers for enabling mass copyright infringement.
Noncommercial use is not well established in copyright law, which is the law that actually matters. I know other forms of law actually do establish noncommercial and commercial use standards, but copyright does not recognize them.
As for "Facebook won't sue"? Sure, except we don't have to worry about just Facebook. We have to worry about anyone with a derivative model. There's an entire industry of copyleft trolls[0] that could construct copyright traps with them.
Individuals can practically ignore NC mainly because individuals can practically ignore most copyright enforcement. This is for the same reason why you can drive 55 in a 30mph zone and not get a citation. It's not that speeding is now suddenly legal, it's that nobody wants to enforce speed limits - but you can still get nailed. The moment you have to worry about NC, there is no practical way for you to fit within its limits.
Noncommercial licenses are taken up in "GREAT MINDS v. FEDEX OFFICE AND PRINT SERVICES, INC 886 F.3d 91 (2nd Cir. 2018). Thé court explains they are enforceable and are basically just a category of contract. So, as long as the contract is clear, it’s probably enforceable.
> if you don't think that all of these situations are equally infringing of CC-BY-NC, then you need to explain why some are commercial and some are not. Keep in mind that every exception you make can be easily exploited to strip the NC clause off of the license.
You're right: those are all equally infringing CC-BY-NC. I don't see a problem.
> My point is that if you don't think that all of these situations are equally infringing of CC-BY-NC, then you need to explain why some are commercial and some are not.
What “NonCommercial” means in the license is explictly defined in the license, and if you think either those examples, or more to the point, every possible use ever so as to render ‘NonCommercial’ into ‘no use’ as you have claimed, you need to make that argument, based on the definition in the license, not some concept of what might be construed as commercial use by general legal principles if the license used the term without its own explicit definition.
My understanding (IANAL) [1] is that copyright licenses have no say on the output of software. Further, CC licenses don't say anything about running or using software (or model weights). It's therefore questionable whether the CC-BY-NC license actually prevents commercial use of the model.
You're correct, but no one has had the balls (or the lawyers) to clarify this in court yet. Expect to see hosting providers complying with takedown requests for the foreseeable future.
I don't remember the details (or outcome) but there was a lawsuit a few years ago involving CAD or architecture software and whether they could limit how the output images were used because they were assemblages of clipart that the company asserted were still protected by copyright. Something like that. A lot of "AI" output potentially poses a similar issue, just at a far more granular level.
> "Meta is really clearly trying to differentiate themselves from OpenAI here. Open source + driving home "we don't use data we haven't paid for / don't own"."
Isn't Meta settling lawsuits for this right now? In addition to violating user privacy (another lawsuit)...
Meta is attempting to destroy competition; that's it. Similar to how they paid a fortune to lobby against Tiktok for the exact reasons Meta is under active investigation (again). The irony.
It's likely partly a PR/branding exercise as well.
In the new world that Meta sees, of VR/AR and AI, Meta is in a position already were people don't want them to have much power in this world, because they don't trust them over privacy etc, meta is trying to pivot to become more trustworthy so they make genuine moves in this space.
That, or this is an ongoing research lab (FAIR) that has existed for ~half a decade and has advanced the state-of-the-art in AI further than Apple, Microsoft and Google combined.
If all three of those companies have something to show for their research, none of it is at the scale or level of accessibility Pytorch, Llama and now Audiocraft offer.
> all of which are also spending a fuck load on internal AI research.
But their internal research stays internal. Sometimes, they put out "papers" which are glorified advertisements, often going as far as hiding the model architecture just to keep their competitive advantage.
The fact that Meta is able to lie and call their restrictive licensing open source is nearly as misleading as "OpenAI."
We need to do better than to repeat these claims uncritically. The weight licenses are not "open source" by any useful definition, and we should not give Meta kudos for their misleading PR (especially considering that they almost surely ignored any copyright when training these things - rules for thee, but not for me).
"Not as closed as OpenAI" is accurate, but also damning with faint praise.
Meta says to imagine you can: "Imagine a professional musician being able to explore new compositions without having to play a single note on an instrument. Or an indie game developer populating virtual worlds with realistic sound effects and ambient noise on a shoestring budget. Or a small business owner adding a soundtrack to their latest Instagram post with ease."
Research does exist you know. This is immensely helpful for a huge number of people in academia.
If you want to build a company, perhaps you should do what everyone in the industry has done for millennia, copy the movements performed and optimize them while doing so.
Just some general piece of advice: it's not productive to constantly be giving out the worst criticism you possibly can when someone does something that's not terrible but still unacceptable. Doing so just tells the companies that nothing satisfies the community and that they should stop trying.
Instead, it's better to mention what they did right and point to how they can make it better.
They are doing PR damage control with an influx of AI stuffs due to the ridicule of metaverse and the recent revelations of threads (for which they are playing the long AI game) -- [are not concrened about all threads and IG and other accounts being linked via their internal LLMs we will never hear about?
Yes. Meta is in the business of commanding as much of peoples time as possible. AI is more or less the biggest danger to this model (apart from legislation, theoretically, but let's not kid ourselves). Making AI a commodity is in their very interest.
They're not, they're playing a longer Microsoft style game to corrupt the meaning of open source, and releasing models under their terms to undermine competitors.
Nah, this is just the modern tech playbook:
First you open source stuff, then you can monitor all the related development happening and whenever you see areas of interest/popularity, you simply clone the functionality or buy out whatever entity is building that interesting stuff.
You can - as long as it's not commercial. It's a broad definition, but a good eule of thumb is if you're not directly making money out of the generated audio. They may still cone for you if you're making money indirectly, so consult a lawyer.
I can see some fantastic uses for this in generating complex acoustic environments to layer over TTS or real recordings for speech-to-text model training. I wonder if that is occupying some kind of gray-area. For example you have 1000hrs of clean speech from the librispeech corpus. It would be trivial to use this tool and available weights to generate background noise, environmental noise and the like, and then layer this with the clean speech to cheaply train a much more robust model. The environmental audio you create would never be directly shared or sold, but it would impact the overall quality of the STT model that you train from the combined results.
Can I haz full version of Bach + `An energetic hip-hop music piece, with synth sounds and strong bass. There is a rhythmic hi-hat patten in the drums.` please?
The difference between MBD-EnCodec and EnCodec is pretty interesting. MBD variant sounds more like a professional studio recording, while the EnCodec feels like a richer sound.
AudioGen seems really fascinating. I have some dumb questions.
While the datasets used for training AudioGen aren't available, is there any kind of list where one can review the tags or descriptions of the sounds on which the model was trained? Otherwise how do you know what kinds of sounds you can reasonably expect AudioGen to be capable of generating? And what happens if you request a sound which is too obscure or something not found in the dataset?
What are AudioGen's capabilities regarding spatial positioning? First example: can it generate a siren that starts in front and moves left to right and complete a full circle around the listener? Second example: can it do the same siren but on the Y axis, so it start at the front, it goes over the listener and then it goes under them to complete the circle?
> It’s unlikely that model weights can be copyrighted, as they’re the result of an automatic process.
If they can’t for that reason alone, then the model is a mechanical copy of the training set, which may be subject to a (compilation) copyright, and a mechanical copy of a copyright-protected work is still subject to the copyright of the thing of which it is a copy.
OTOH, the choices made beyond the training set and algorithm in any particular training may be sufficient creative input to make it a distinct work with its own copyright, or there may be some other basis for them not being copyright protected. But the mechanical process one alone just moves the point of copyright on the outcome, it doesn’t eliminate it.
That isn't necessarily true. You're saying that the model weights would be a derivative work. Derivative work is insufficiently transformative to fully distinguish the output from the input.
In this instance, it's very likely that the process is sufficiently transformative. A set of model weights look nothing like the Mona Lisa, nor can they be directly transformed back into it. What it is NOT is the product of a creative process on the part of a human, and is thus ineligible for copyright.
It is as though we are able to distill meaning using an automatic process. Copyright doesn't protect meaning, only expression, and it only protects expressions that were generated by humans.
The network itself might be patentable if it isn't obvious.
If a network was copyrighted and it was found that the function of the network was inseparable from its expression, it would no longer be eligible for copyright on that basis.
I think people often misunderstand the application of copyright to GPL code in this way.
Now, will any of this stop people from CLAIMING copyright? No. It'll have to be fought out in courts.
Diffusion model is now SOTA in audio and image generation. Has anyone given it a shot on texts?
Audio is more similar to language than images because of more stronger time dependency.
The paper says the critical step they took for making diffusion model work for audio was splitting the frequency bands and applying diffusion separately to the bands (because full band model had limitations due to poor modeling of correlations between low frequency and high frequency features).
I think something could be done on text side as well.
There are two problems with this. Diffusion models work on a single rule of thumb: if you keep adding small, noisy gaussian steps to a "nice" distribution many times, you get uniform gaussian at the end.
So, for text: a) what is the equivalent of a small, noisy step? and b) what is the equivalent of a uniform gaussian in language space?
If you can solve a and b, you can make diffusion work for text, but there hasn't been any significant progress there afaik.
I'm looking forward to playing with the M1 Mac apps/cli-tools that will probably come out for this in the next week or so! Being able to run this stuff locally is a lot of fun.
I've run Stable Diffusion locally (both from the cli and later using GUI wrappers) and that used my GPUs, I've also run Llama locally but I believe that was on the CPU (I used both llama.cpp, cli, and Ollama, gui). So to sum it up: yes? Or at least it's good enough for me.
"Now, increasingly, we live in a world where more and more of these cultural artifacts will be coming from an alien intelligence. Very quickly we might reach a point when most of the stories, images, songs, TV shows, whatever are created by an alien intelligence.
And if we now find ourselves inside this kind of world of illusions created by an alien intelligence that we don’t understand, but it understands us, this is a kind of spiritual enslavement that we won’t be able to break out of because it understands us. It understands how to manipulate us, but we don’t understand what is behind this screen of stories and images and songs."
Maybe this us vs them mentality is the biggest bottleneck.
If instead you consider that this new form of 'alien' intelligence is actually a descendant of human intelligence, that we are raising a new species which will inherit what humans have built (ideally only the good parts) and then improve upon it further..
It may sound grandiose, but that perspective changes everything.
"generating new music in the style of existing music" will probably be a huge field soon. I can't wait for it to happen, it's a low-cost way of producing even more music to listen to.
> I can't wait for it to happen, it's a low-cost way of producing even more music to listen to.
I can't really understand this. I'm a DJ and a huge music nerd, and I spend a lot of time every week discovering new music from the past 100 years and all over the world, and I'm constantly struck by _how much of it there is_. I've spent weeks just digging through psych-funk records from West Africa from the 1970s.
How can you have the impression we're so desperate for more music that we need computer programs to generate it for us?
Yes but it's not really fungible. My favourite artist is Fats Waller and they don't make anything like that anymore. Most people are only interested in some category of music, not all of it.
Music is self-expression. I don’t always identify entirely with others. I always identify with my self. Having music generated for you on such a personalized level is an attractive prospect.
I don’t think this replaces “100% organic, human-made” music, though. I think there’ll always be a reason to listen to music made by other people. But I think this changes the landscape of how and why people create music to begin with. It certainly will devalue existing music, since everyone has something they may prefer that they can generate instantly.
I think generative AI is a terrible technology for artists who want to make money from their art, but in my personal opinion, I strive for a world where art isn’t a transaction, but a gift of human expression and connection. A world where art is appreciated for the emotion, stories, and ideas it conveys rather than the monetary value it holds. Generative AI might disrupt the traditional economic models in the art world, but it also opens up new opportunities for creative exploration and personal expression. It’s a challenging evolution, but one that could potentially democratize art, making it more accessible and personal than ever before! Bring on the Renaissance: Part 2!
> I strive for a world where art isn’t a transaction, but a gift of human expression and connection. A world where art is appreciated for the emotion, stories, and ideas it conveys rather than the monetary value it holds.
In a world where nobody is compensated for their art, the only people making art will be the ones privileged enough to have the means to do so for free. I don't see how this leads to "Renaissance: Part 2."
I don’t agree with your definition of music. For me, as both a musician and a listener, music is communication between human beings via harmonic carrier waves.
Using a machine to make word salad copies of existing communiques is literally just nonsense to me
There is a lot of human music for sure, great music from all eras, but just the other day i generated a song which was pure crystal harp. So, how many songs of crystal harp are out there? 1000 all n all? 10.000 maybe? Now i can generate one thousand crystal harp songs per day.
What if I generated a music of 12 billion farts, how many asses are out there ? 7 billions all in all maybe ? Now I can generate one billions of fart songs per day
I came to understand a very large portion of the population just wants content, any type, any quality, to fill the void. They'll consume anything as long as it's new. Content to fill the empty vessels we became. Just look around, mainstream music, movies, podcasts, news, it's mostly mediocre, but it goes real fast, you get new mediocrity delivered every day
Musician here. While I agree with you that there is a nearly endless heap of music to dig through, I think it's interesting to think about the possibility of hearing genre crossovers and styles that don't yet exist.
As an aside, a lot of musicians seem to dislike this kind of technology, but I never saw music as a competition. I don't care if some inexperienced kid is generating bangers from his bedroom even though he can't play a single instrument. It's just something else to listen to. I write music for me.
It’s the same reason we have 250 Marvel movies that all tell the same story. People want the same, but different. They don’t give a damn about human creativity for the most part.
Yes. Installation instructions on the front of the repo, then click on the model readme for sample getting started code (10 lines of python and you get output.)
This is Spotify's route to profitability - the Netflix model of generating their own "content" (/music), and not having to pay the labels. Premium plans for us music nerds who want a human at the other end, regular plans for plebs who just want to fill the silence with something agreeable.
Although I think AI-generated and AI-augmented (using voice cloning, etc.) artists are a given, for Spotify to stop paying labels they'd have to be able to remove all non-Spotify content from their streaming catalog. That doesn't seem like a possibility in our lifetimes. (Also, Spotify hasn't even been able to build a sustainable business on podcasts, which they copy to their closed platform for free.)
It's an interesting thought experiment, though. I can imagine that "environmental audio" companies like Muzak have about 5 years left before they either adapt or die. What other kinds of companies are in trouble?
Their current pay structure is royalties, i.e. per listen. If they can route their audience to mostly AI generated content in time (say, 5-10 year transition), and it's just as good for most people, then they can negotiate much lower prices with the labels. We all grumble about Netflix being full of junk, but most of us are still subscribers, despite a sparse catalog of big name movies.
> then they can negotiate much lower prices with the labels
Or alternatively, if the labels are not stupid, they'll negotiate for a higher price per listen (or similar), as they are still as essential to the service as before.
As an amateur musician I’m wondering if there are any of these audio generators that you can give a tune or chord progression to riff on. ABC format maybe? There are lots of folk tunes on thesession.org.
Could you generate a rhythm track? Ideally you could make songs one track at a time, by giving it a mix of the previous tracks and asking it to make another track for an instrument. Or, give it a track and ask it to do some kind of effect on it.
Another interesting use might be generating sound samples for a sampled instrument.
If you mean to give a source of melody of 30sec and extend that melody into a full song, yes MusicGen can do that. There are two ways to extend a song based on a melody: 1) give a sample, and continue the song from that sample as close as possible, and 2) give a melody as an inspiration.
They both work in varying degrees of success. Audiocraft on github, issues or discussion sections have a lot of questions answered.
Did they change the base model? If not, then the audiocraft_plus, is based on audiocraft, and it creates music of length close to 5 minutes.
I don't know if audiocraft_plus incorporates all three modalities of the release, MusicGen, AudioGen, and EnCodec. It uses MusicGen for sure, all four models, small, medium, large and melody.
Yes, of course it is more than that. You can create full songs of several minutes. I have published 20 songs on youtube of 2-3 minutes each. Listen to that song, the best i have made so far [1]. It doesn't repeat at all!
I haven't looked closely to this release, is the audiocraft page on github different than facebookresearch/audiocraft? The other two modalities may be new, AudioGen, and EnCodec, but i was under the impression that they changed the license to full open source, and that was that.
Thanks. What I hear there is 19s of material on repeat, with the parallel parts variously brought in and out, and no other variation except in decorative detail.
So yes not a literal repeat, but no not enough to rate as a "full song", IMO.
Plus it is not actually song at all, note. Singing would particularly expose its repeat-and-vary trick.
Coqui-TTS with vtck/vits is very good right now. Not as good as eleven labs or coqui studio, but for fast open TTS it's pretty good, in case you're not familiar with it.
It will be great when there's eventually something open that competes with the closed models out there.
In my opinion, the Free Software Foundation, ironically, since they invented the movement, with open source starting out as a tacky rip-off with the ethics stripped out. After decades, open source converged on free software.
The people who created the term: the Open Source Initiative.
Before, people most often used "free software" as defined by the free software movement, but some disliked this term because it's confusing (most think "free" means no money) and perceived to be anti-commercial.
The term "open source software" was chosen and given a precise definition.
It's dishonest, then, for people to use the term "open source software" with a different interpretation when it was specifically chosen to avoid confusion.
> It's dishonest, then, for people to use the term "open source software" with a different interpretation
I disagree. You're saying that they "invented" the term, but it's a very generic term. The source is open, it's open source. I bet people were using the term before they claim they invented it.
In that context, it is very fine to use a different definition and in fact here's my definition, and I guess most people (maybe not on HN) share it: if the source is visible by the general public, it's open source.
> Our audio research framework and training code is released under the MIT license to enable the broader community to reproduce and build on top of our work
It's pretty common in academic research for trained model weights to be licensed under something different from the code that one would run to create such a model if one had both sufficient compute resources and the same training dataset. That is, if those weights are ever released at all!
IMO, while I'd rather have one part permissively licensed than nothing at all... it stinks that companies sponsoring researchers get an un-nuanced level of street cred for "open sourcing" something that they know nobody will ever be able to reproduce because their data set and/or their compute grid's optimizations are proprietary.
As it stands, I'm not at all sure that the outputs of this model can be used for commercial videos.
334 comments
[ 3.3 ms ] story [ 262 ms ] threadhttps://audiocraft.metademolab.com/audiogen.html
The samples included in the press release are quite impressive to my ears, but the other samples (especially from AudioGen) have a hint of artificially.
As usual the music is quite repetitive, but I'm looking forward to tools that simplify changing the prompt whilst it generates over a window. I can only imagine the consequences for royalty free music.
Edit: the "Text-to-music generation with diffusion-based EnCodec" samples are quite impressive.
https://gwern.net/complement
If they just keep their models, people won't be interested and will build over ChatGPT or Bard.
That said, there are a ton of “look at this cool thing out research team did” and then you never hear about them again things from Google. They even built a music generator that was closed to the public until recently.
https://blog.google/technology/ai/musiclm-google-ai-test-kit...
They are trying to kill the market before they get left out.
If you're a company and wanted to integrate an LLM into your product if the choice is between several equally good models, but one is free and open-source which would you pick?
Aside from keeping competition at bay, this move also gives Meta leverage because ecosystems are now being built around their projects. If these models see wide-scale adoption they could later launch AudioCraft+ as a licensed version with some extra features for example.
Alternatively, they might offer support or hosting for their open source projects.
Right now though I think the primary benefit of these open sourced models is to attract talent. If Meta is seen as one of the leaders in AI then researchers will want to work for them simply for the prestige.
Arguably one of the reasons Meta has been behind so many awesome projects like PyTorch and React over the last decade was because they were seen as the cool place for recently graduated, but talented software engineers to work in ~2010.
I wondered though, generative AI is hurling us into a world where we'll need more mechanisms to sort real from fake, provenance will play a large part, and meta's platforms could be part of the answer. i.e. content linked to actual verifiable people.
He touches on it briefly in this podcast episode: https://www.therobotbrains.ai/who-is-yann-lecun
That'll make musicians happy with big tech as well, just like artists are. *sigh*
Then use the user’s action to iteratively refine your classification, until you end up with something tailor-made.
I suppose they might try, anyway.
This is precisely the opposite of the context I was remarking on.
Also it doesn't really matter on what the law says, RIAA in the last iteration, relied on the fact it you would rather pay a fine than be able to afford expensive lawyers to fight the specifics out in court on average.
It was always about disproportionate ability to bring resources against individual "offenders" to create fear among everyone to deter "undesirable" forms of copying, not necessarily what the legal protections were.
---
[1] Unless you specifically commissioned it under a contract which gave you the right
[2] See recent cases including those related to Kris Kashtanova and Andy Warhol.
[3] Not the song, just the performance, aka Taylor Swift version, for good explanation of how the rights are divvied up in the music industry a Planet Money series covers it well https://www.npr.org/sections/money/2022/10/29/1131927591/inf...
Also, IIRC, RIAA did not bring many resources to bear against e.g. "home taping" itself, because they could essentially never know that it had occured. The overwhelming majority of their efforts went into trying to takedown people distributing multiple copies.
The Kashtanova case does not cover derivation in any real way, but is really about copyright attribution choices between human and software.
The Warhol case specifically tests a fair use claim, not a derivation claim.
https://en.wikipedia.org/wiki/Home_Taping_Is_Killing_Music
Recording industries have fought end user reproduction often. They’ve fought sampling battles.
Go after the pocketbooks and go after the technology waves. If there’s a derivative argument they can make, they will.
> "This exception was crucial in RIAA v. Diamond Multimedia Systems, Inc.,[14] the only case in which the AHRA's provisions have been examined by the federal courts. The RIAA filed suit to enjoin the manufacture and distribution of the Rio PMP300, one of the first portable MP3 players, because it did not include the SCMS copy protection required by the act, and Diamond did not intend to pay royalties. The 9th Circuit, affirming the earlier District Court ruling in favor of Diamond Multimedia,[15] ruled that the "digital music recording" for the purposes of the act was not intended to include songs fixed on computer hard drives. The court also held that the Rio was not a digital audio recording device for the purposes of the AHRA, because 1) the Rio reproduced files from computer hard drives, which were specifically exempted from the SCMS and Royalty payments under the act, 2) could not directly record from the radio or other transmissions. "
From the AHRA itself:
> No action may be brought under this title alleging infringement of copyright based on the manufacture, importation, or distribution of a digital audio recording device, a digital audio recording medium, an analog recording device, or an analog recording medium, or based on the noncommercial use by a consumer of such a device or medium for making digital musical recordings or analog musical recordings.
and from Wikipedia again:
> In regard to home taping, the provision broadly permits noncommercial, private recording to analog devices and media. However, it fails to resolve the home taping debate "conclusively," as it only permits noncommercial, private recording to digital devices and media when certain technology is used.
> Two reports by the House of Representatives characterize the provision as legalizing digital home copying to the same degree as analog. One states "in the case of home taping, the exemption protects all noncommercial copying by consumers of digital and analog recordings,"[22] and the other states "In short, the reported legislation [Section 1008] would clearly establish that consumers cannot be sued for making analog or digital audio copies for private noncommercial use."[23]
Similarly, language in the RIAA v. Diamond Multimedia decision suggests a broader reading of the Section 1008 exemptions, providing blanket protection for "all noncommercial copying by consumers of digital and analog musical recordings" and equating the spaceshifting of audio with the fair use protections afforded home video recordings in Sony v. Universal Studios:
>> In fact, the Rio's operation is entirely consistent with the Act's main purpose – the facilitation of personal use.
They went after ISPs and torrent sites which only hosted magnet links and many others who shouldn’t have been really sued
The goal was to create a very hostile environment for downloading songs to protect their interests - “you wouldn’t download a car!”
It was never the goal nor ever realistic to actually pursue enforcement action against every offender , the idea was to change behavior with all the related actions .
They did end up changing behavior, people just didn’t want the hassle, or be in fear so paying for streaming for access had a stronger value proposition, it was not what the RIAA planned , but benefits enormously today.
True, but only because you have that right anyway. I can do anything I like with copyrighted content I legally possess, as long as I don't distribute the results of my efforts.
If you purchase the music, you have a (sometimes explicit, sometimes implicit) license to do certain things with the music, entirely independent of any concept of "fair use".
The question is not "is training part of fair use?" but "is training part of, implicitly or explicitly, the rights I already have after purchase?"
Given that "training" can be done by simple playing the music in the presence of a computer with its microphone turned on, it's not clear how this plays out legally.
- First sale: The Takings Clause prohibits government theft of private property without compensation. Because copyright owners are using a government-granted monopoly to enforce their rights, we have to bound those rights to avoid copyright owners being able to just come and take copies of books or music you've lawfully purchased.
- Fair use: The 1st Amendment prohibits government prohibitions on free speech. Because copyright owners are using a government-granted monopoly to enforce their rights, we have to bound those rights to avoid copyright owners being able to censor you.
If you hinge your argument on "I bought a copy", you're making a first sale argument.
Notably, first sale is limited to acts that do not create copies. This limit was established by the ReDigi case[0]. Copyright doesn't care about the total number of copies in circulation, it cares about the right to create more. So an AI training defense based on first sale grounds would fail because training unequivocally creates copies.
Fair use, on the contrary, does not care if you bought a copy of a work legally. It only cares about balancing your right to speech against the owners' right to a monopoly over theirs. And it has so far been far more resistant to creative industry attempts to limit exceptions to copyright - to the point where I would argue that "fair use" is an effective shorthand for any exception to copyright, including ones in countries that have no fair use doctrine and do not respect judicial precedent.
The courts won't care how the training comes about, just if the act of training an AI alone[1] would compete with licensing the images used in the training set data.
[0] https://en.wikipedia.org/wiki/Capitol_Records,_LLC_v._ReDigi....
[1] Notably, this is separate from the act of using the AI to generate new artistic works, which may be infringing
> Mitigations: Vocals have been removed from the data source using corresponding tags, and then using a state-of-the-art music source separation method, namely using the open source Hybrid Transformer for Music Source Separation (HT-Demucs).
> Limitations: The model is not able to generate realistic vocals.
(https://github.com/facebookresearch/audiocraft/blob/main/mod...)
I suspect this was a combination of playing it safe and that the model isn't well architected to reproduce meaningful vocals.
Seriously, you couldn't sell this output for a free mobile clicker game.
Bruh, music is subjective as hell, and I can already tell I hate this song.
Meta is really clearly trying to differentiate themselves from OpenAI here. Open source + driving home "we don't use data we haven't paid for / don't own".
Furthermore, MusicGen's weights are licensed CC-BY-NC, which is effectively a nonlicense as there is no noncommercial use you could make of an art generator[1]. This is not only a 'weights-available' license, but it's significantly more restrictive than the morality clause bearing OpenRAIL license that Stability likes to use[2].
[0] https://github.com/facebookresearch/llama/blob/main/MODEL_CA...
[1] https://github.com/facebookresearch/audiocraft/blob/main/LIC...
[2] These are also very much Not Open Source™ but the morality clauses in OpenRAIL are at least non-onerous enough to collaborate over.
How do you figure? Have you never just...made stuff to make stuff?
Obviously, you can't host a commercial art generation service with a noncommercial-use license, and (insofar as art produced by a generator is a derivative work of the model weights, which is a controversial and untested legal theory) you can’t make commercial art with a noncommercial license, but not all art is commercial.
You're probably thinking of "not charging a fee to use", which is a subset of all the ways you can monetize a creative work. You can still make money off of AudioCraft by just hosting it with banner ads next to the output. Even a "no monetization" clause[0] would be less onerous than "noncommercial use only", because it'd at least be legal to use AudioCraft for things like background music in offices.
[0] Which already precludes the use of AudioCraft music on YouTube since you can't do unmonetized uploads anymore
The definition of “NonCommercial”, the oddly capitalized term of art in the license, is not a matter of general law, it is a matter of the license, which defines it as “not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange.”
> Even if you don’t intend to make money the law still considers the work itself to be commercial.
Even if you do make money, if the use is “not primarily intended” for that purpose, it is "NonCommercial" in the terms of the license.
> That’s why CC-BY-NC has to have a special “filesharing is non-commercial” statement in it, because people have made successful legal arguments that it is.
It has the filesharing term in it because it permits that particular exchange-of-value as a primary purpose.
> Even a “no monetization” clause would be less onerous than "noncommercial use only"
How would a clause that prohibits monetization entirely be less onerous than one which prohibits it only as the primary intent of use?
> it’d at least be legal to use AudioCraft for things like background music in offices.
It is legal to use it for that purpose (in a for-profit enterprise, I suppose, one might make an argument that any activity was ultimately primarily directed at “commercial advantage”, but in a government or many nonprofit environments, that wouldn’t be the case.)
I realize, this isn't legal advice, YMMV, etc.
A resort, probably not, ambiance is, at least arguably, a marketable commercial advantage; a private club in the “mutual benefit organization” sense (rather than a “business selling memberships”, which is just like a resort), probably, because their interest, even indirectly, isn’t making money.
- If I use AudioCraft to post freely-downloadable tracks on my SoundCloud, I still get the benefit of having a large audio catalog in my name, even if I'm not selling the individual tracks. I could later compose tracks on my own and ride off the exposure I got from posting "noncommercially".
- If I run AudioCraft as a background music generator in my store, I save money by not having to license music for public performance.
- If I host AudioCraft on a website and put ads on it, I'm making money by making the work available, even though I'm not charging a fee for entry.
I suspect that a lot of people reading this are going to have different arguments for each. My point is that if you don't think that all of these situations are equally infringing of CC-BY-NC, then you need to explain why some are commercial and some are not. Keep in mind that every exception you make can be easily exploited to strip the NC clause off of the license.
If you're angry at the logic on display here, keep in mind that this is how judges will construe the license, and probably also how Facebook will if you find a way to make any use of their AI. The only thing that stops them from rugpulling you later is explicit guidance in CC-BY-NC. Unfortunately, the only such guidance is that they don't consider P2P filesharing to be a commercial use.
So, absent any other clarifications from Facebook, all you can do without risking a lawsuit is share the weights on BitTorrent.
EDIT: And yes, I have made stuff just to make stuff. I license all of that under copyleft licenses because they express the underlying idea of 'noncommercial' better than actual noncommercial clauses do.
Do you think that non commercial use simply doesn't exist or something?
Because non commercial use isn't some crazy concept. It is a well established one, that doesnt disclude literally everything.
Also, you are ignoring the idea that Facebook will almost certainly not sue anyone for using this for any reason, except possibly Google or Apple.
So if you aren't literally one of those companies you could probably just use it anyway, ignore the license completely, and have zero risk of being sued.
http://esr.ibiblio.org/?p=4559
Whatever happened to esr? Did he just get too paranoid and clam up?
I presume you mean in USA, because in UK you don't have a general private right to copy. Our "Fair Dealing" is super restrictive compared to Fair Use.
I like that it makes software like iTunes contributory infringers for enabling mass copyright infringement.
As for "Facebook won't sue"? Sure, except we don't have to worry about just Facebook. We have to worry about anyone with a derivative model. There's an entire industry of copyleft trolls[0] that could construct copyright traps with them.
Individuals can practically ignore NC mainly because individuals can practically ignore most copyright enforcement. This is for the same reason why you can drive 55 in a 30mph zone and not get a citation. It's not that speeding is now suddenly legal, it's that nobody wants to enforce speed limits - but you can still get nailed. The moment you have to worry about NC, there is no practical way for you to fit within its limits.
[0] https://www.techdirt.com/2021/12/20/beware-copyleft-trolls/
No, for “NonCommercial”, what actually matters is the explicit definition in the license.
Noncommercial licenses are taken up in "GREAT MINDS v. FEDEX OFFICE AND PRINT SERVICES, INC 886 F.3d 91 (2nd Cir. 2018). Thé court explains they are enforceable and are basically just a category of contract. So, as long as the contract is clear, it’s probably enforceable.
You're right: those are all equally infringing CC-BY-NC. I don't see a problem.
> this is how judges will construe the license
What “NonCommercial” means in the license is explictly defined in the license, and if you think either those examples, or more to the point, every possible use ever so as to render ‘NonCommercial’ into ‘no use’ as you have claimed, you need to make that argument, based on the definition in the license, not some concept of what might be construed as commercial use by general legal principles if the license used the term without its own explicit definition.
[1] https://opensource.stackexchange.com/questions/12070/allowed...
I suppose your point would stand if the software were a quine?
I'm sorry, what?
r/stablediffusion gives you a hundred examples daily of people just having fun and not thinking of monetizing their generations
Isn't Meta settling lawsuits for this right now? In addition to violating user privacy (another lawsuit)...
Meta is attempting to destroy competition; that's it. Similar to how they paid a fortune to lobby against Tiktok for the exact reasons Meta is under active investigation (again). The irony.
In the new world that Meta sees, of VR/AR and AI, Meta is in a position already were people don't want them to have much power in this world, because they don't trust them over privacy etc, meta is trying to pivot to become more trustworthy so they make genuine moves in this space.
But their internal research stays internal. Sometimes, they put out "papers" which are glorified advertisements, often going as far as hiding the model architecture just to keep their competitive advantage.
https://www.americanbar.org/groups/science_technology/public...
We need to do better than to repeat these claims uncritically. The weight licenses are not "open source" by any useful definition, and we should not give Meta kudos for their misleading PR (especially considering that they almost surely ignored any copyright when training these things - rules for thee, but not for me).
"Not as closed as OpenAI" is accurate, but also damning with faint praise.
In reality, you can't, as they licensed the weights for noncommercial use only: https://github.com/facebookresearch/audiocraft#license
If you want to build a company, perhaps you should do what everyone in the industry has done for millennia, copy the movements performed and optimize them while doing so.
[0]: https://github.com/facebookresearch/audiocraft#models
/edit On a more serious node. I already see the 24/7 lofi girl streaming generated music. The sample[1] on lofi sounds pretty good.
[1]https://dl.fbaipublicfiles.com/audiocraft/webpage/public/ass... "Lofi slow bpm electro chill with organic samples"
Samplers have been around since the 70s.
Can I haz full version of Bach + `An energetic hip-hop music piece, with synth sounds and strong bass. There is a rhythmic hi-hat patten in the drums.` please?
(https://dl.fbaipublicfiles.com/audiocraft/webpage/public/ass...) ?
Curious if I’m alone in that.
(At the bottom https://audiocraft.metademolab.com/musicgen.html)
For what it’s worth though, the voice based examples sound dramatically better with MBD
https://audiocraft.metademolab.com/encodec.html
While the datasets used for training AudioGen aren't available, is there any kind of list where one can review the tags or descriptions of the sounds on which the model was trained? Otherwise how do you know what kinds of sounds you can reasonably expect AudioGen to be capable of generating? And what happens if you request a sound which is too obscure or something not found in the dataset?
What are AudioGen's capabilities regarding spatial positioning? First example: can it generate a siren that starts in front and moves left to right and complete a full circle around the listener? Second example: can it do the same siren but on the Y axis, so it start at the front, it goes over the listener and then it goes under them to complete the circle?
The code is MIT, though.
If they can’t for that reason alone, then the model is a mechanical copy of the training set, which may be subject to a (compilation) copyright, and a mechanical copy of a copyright-protected work is still subject to the copyright of the thing of which it is a copy.
OTOH, the choices made beyond the training set and algorithm in any particular training may be sufficient creative input to make it a distinct work with its own copyright, or there may be some other basis for them not being copyright protected. But the mechanical process one alone just moves the point of copyright on the outcome, it doesn’t eliminate it.
In this instance, it's very likely that the process is sufficiently transformative. A set of model weights look nothing like the Mona Lisa, nor can they be directly transformed back into it. What it is NOT is the product of a creative process on the part of a human, and is thus ineligible for copyright.
It is as though we are able to distill meaning using an automatic process. Copyright doesn't protect meaning, only expression, and it only protects expressions that were generated by humans.
The network itself might be patentable if it isn't obvious.
If a network was copyrighted and it was found that the function of the network was inseparable from its expression, it would no longer be eligible for copyright on that basis.
I think people often misunderstand the application of copyright to GPL code in this way.
Now, will any of this stop people from CLAIMING copyright? No. It'll have to be fought out in courts.
Audio is more similar to language than images because of more stronger time dependency.
The paper says the critical step they took for making diffusion model work for audio was splitting the frequency bands and applying diffusion separately to the bands (because full band model had limitations due to poor modeling of correlations between low frequency and high frequency features).
I think something could be done on text side as well.
So, for text: a) what is the equivalent of a small, noisy step? and b) what is the equivalent of a uniform gaussian in language space?
If you can solve a and b, you can make diffusion work for text, but there hasn't been any significant progress there afaik.
And if we now find ourselves inside this kind of world of illusions created by an alien intelligence that we don’t understand, but it understands us, this is a kind of spiritual enslavement that we won’t be able to break out of because it understands us. It understands how to manipulate us, but we don’t understand what is behind this screen of stories and images and songs."
-Yuval Noah Harari
If instead you consider that this new form of 'alien' intelligence is actually a descendant of human intelligence, that we are raising a new species which will inherit what humans have built (ideally only the good parts) and then improve upon it further..
It may sound grandiose, but that perspective changes everything.
I can't really understand this. I'm a DJ and a huge music nerd, and I spend a lot of time every week discovering new music from the past 100 years and all over the world, and I'm constantly struck by _how much of it there is_. I've spent weeks just digging through psych-funk records from West Africa from the 1970s.
How can you have the impression we're so desperate for more music that we need computer programs to generate it for us?
I don’t think this replaces “100% organic, human-made” music, though. I think there’ll always be a reason to listen to music made by other people. But I think this changes the landscape of how and why people create music to begin with. It certainly will devalue existing music, since everyone has something they may prefer that they can generate instantly.
I think generative AI is a terrible technology for artists who want to make money from their art, but in my personal opinion, I strive for a world where art isn’t a transaction, but a gift of human expression and connection. A world where art is appreciated for the emotion, stories, and ideas it conveys rather than the monetary value it holds. Generative AI might disrupt the traditional economic models in the art world, but it also opens up new opportunities for creative exploration and personal expression. It’s a challenging evolution, but one that could potentially democratize art, making it more accessible and personal than ever before! Bring on the Renaissance: Part 2!
In a world where nobody is compensated for their art, the only people making art will be the ones privileged enough to have the means to do so for free. I don't see how this leads to "Renaissance: Part 2."
this isn't democratizing art, and i would argue it has nothing to do with art. it is giving us an endless faucet of content, but not art.
Recorded music is the worst thing that happened to music.
I came to understand a very large portion of the population just wants content, any type, any quality, to fill the void. They'll consume anything as long as it's new. Content to fill the empty vessels we became. Just look around, mainstream music, movies, podcasts, news, it's mostly mediocre, but it goes real fast, you get new mediocrity delivered every day
As an aside, a lot of musicians seem to dislike this kind of technology, but I never saw music as a competition. I don't care if some inexperienced kid is generating bangers from his bedroom even though he can't play a single instrument. It's just something else to listen to. I write music for me.
It's an interesting thought experiment, though. I can imagine that "environmental audio" companies like Muzak have about 5 years left before they either adapt or die. What other kinds of companies are in trouble?
Or alternatively, if the labels are not stupid, they'll negotiate for a higher price per listen (or similar), as they are still as essential to the service as before.
Could you generate a rhythm track? Ideally you could make songs one track at a time, by giving it a mix of the previous tracks and asking it to make another track for an instrument. Or, give it a track and ask it to do some kind of effect on it.
Another interesting use might be generating sound samples for a sampled instrument.
They both work in varying degrees of success. Audiocraft on github, issues or discussion sections have a lot of questions answered.
I don't know if audiocraft_plus incorporates all three modalities of the release, MusicGen, AudioGen, and EnCodec. It uses MusicGen for sure, all four models, small, medium, large and melody.
https://github.com/GrandaddyShmax/audiocraft_plus
I haven't looked closely to this release, is the audiocraft page on github different than facebookresearch/audiocraft? The other two modalities may be new, AudioGen, and EnCodec, but i was under the impression that they changed the license to full open source, and that was that.
https://www.youtube.com/watch?v=UL1KmrHMjcM
So yes not a literal repeat, but no not enough to rate as a "full song", IMO.
Plus it is not actually song at all, note. Singing would particularly expose its repeat-and-vary trick.
It will be great when there's eventually something open that competes with the closed models out there.
More popular opinion is OSI: https://en.wikipedia.org/wiki/Open_Source_Initiative
They were founded by the persons who (claimed to have) invented the term in order to steward it. It's the same definition as the FSF.
Before, people most often used "free software" as defined by the free software movement, but some disliked this term because it's confusing (most think "free" means no money) and perceived to be anti-commercial.
The term "open source software" was chosen and given a precise definition.
It's dishonest, then, for people to use the term "open source software" with a different interpretation when it was specifically chosen to avoid confusion.
I disagree. You're saying that they "invented" the term, but it's a very generic term. The source is open, it's open source. I bet people were using the term before they claim they invented it.
In that context, it is very fine to use a different definition and in fact here's my definition, and I guess most people (maybe not on HN) share it: if the source is visible by the general public, it's open source.
For what you mean, I use "FLOSS".
The article says this:
> Our audio research framework and training code is released under the MIT license to enable the broader community to reproduce and build on top of our work
IMO, while I'd rather have one part permissively licensed than nothing at all... it stinks that companies sponsoring researchers get an un-nuanced level of street cred for "open sourcing" something that they know nobody will ever be able to reproduce because their data set and/or their compute grid's optimizations are proprietary.
As it stands, I'm not at all sure that the outputs of this model can be used for commercial videos.