Proposal: A Unified Local Directory for LLM Models

I propose that the LLM community agree upon a unified LLM model storage location and organizational subdirectory tree on the major operating systems.

Like many of us, I do a lot of using, testing, and just playing around with LLMs of all types. Each of the tools or packages downloads its models and puts them wherever they like.

This leads to duplicate models in different places on my drives, which wastes space.

Like many people, the fastest most useful drive I have is the main one on my daily driver computer. While I could store the models on a big external drive, I'd be taking a performance hit for tasks that are already in need of all the CPU and/or GPU power I can give them.

Also, although many of the software tools or packages can use the same models, the current "store anywhere" practice means they might not "see" models already downloaded by other softare and "know" that they are available for use.

Additionally, the different LLM software tools and packages may use a different model-storing directory structure. For example, GPT4All stores models in this structure on macOS:

/Users/<your_username>/Library/Application Support/nomic.ai/GPT4All/<model-name>.<suffix>

However, LM Studio stores them like this on macOS:

/Users/<your_username>/Library/Application Support/LM Studio/models/<model-provider-name>/<model-name>/<model-name>.<suffix>

We can change LM Studio's model storage location to point to the GPT4All model directory but LM Studio won't use the GPT4All models in that directory until you "categorize" all the models into LM Studio's preferred directory structure.

That is often the case with LLM software and tools: we can create symbolic links for each software package to a central directory, but if you are constantly trying out new LLM software and tools, that becomes its own hassle as the software and tools may disagree about what all is stored there, how it is named, and what the directory structure should be.

So, my proposal is that the LLM open source community establish a standard location on the top three major operating systems —Linux, macOS, and Windows — that can become the one central place for models to be stored. It could behave like the Documents directory on those operating systems, or like the Desktop — a widely used known location that will be assumed to be there for models to be stored and found.

I suggest ~/Documents/Models but I could see arguments for putting it at the system level for all users to access, such as at /Library/Models on macOS.

Thanks for your consideration.