IDE of the future? Run your LLM model locally with LM Studio
Transforming AI Accessibility: The Rise of Local LLMs
ChatGPT and the rising star of Large Language Models (LLMs) have not only transformed the way we live but also reshaped organizational workflows. As automation becomes increasingly integral to our productivity, adapting to new technological stacks is crucial for survival. However, it might require time and resources to adapt to the new organization transformation 🌟. One of the tools that I recently discovered could help individuals, startups, and even enterprises to transform in that direction is LM Studio. Imagine turning sophisticated AI models into something as accessible as an app on your computer.
LM Studio: Making AI as Easy as Pie 🥧
🤖 LM Studio is a desktop application that allows you to discover, download, and run open-source LLMs. What sets LM Studio apart is its ability to run these models locally on your workstation (basically your laptop), bypassing the need for complex command-line setups. How is that possible? It uniquely accesses major open-source LLM models directly from the Hugging Face — largest ML/DS Hub and runs the models using its platform. Users can explore and select from various open-source models on Hugging Face, with LM Studio simplifying downloading and integrating these models into local environments.
The Evolution of IDEs and LM Studio’s Role 🌍
Remember when GitHub dropped Copilot, and we all went ‘Wow’? The introduction of GitHub’s Copilot marks the beginning of a new era where IDEs will evolve to include functionalities for running LLM models on cloud and local machines. The future of IDEs will likely merge traditional coding environments with AI-assisted tools, enabling developers to leverage various LLM models. Enterprises and smaller businesses will favor local environments. The reasons for that: 1. the cloud is not affordable anymore and 2. privacy matters will prevail. For those familiar with LLMs, this opens the possibility of experimenting with multiple open-source models or even creating custom GPT-type models. At least, this is the logical sequence of where the industry is heading.
Considering the privacy concerns and the need for rapid prototyping, this feature is pivotal at the beginning. And that’s an advantage of platforms like LM Studio. The platform’s user-friendly interface and compatibility with various open-source models streamlined the development process, significantly cutting the time from concept to deployment.
Let me show you how to start with LM Studio. 🎸
LM Studio setup
1️⃣ Download and Instal LM Studio
The installation process is straightforward, making the software accessible even to those with limited technical background. Go to LM Studio and install it for your relevant computer(Mac, Windows or Linux). I have Mac M1, so I installed that one.
2️⃣ Home Page
After you install LM studio, you will open up home page. At home page, you see search button and supporting LLM Models such as LLaMa, Mistral, Falcon etc. If you scroll down, you will see the models and their descriptions. Under the search bar, you will see multiple major open-source model labels such as LLama, Falcon, MPT etc.
Below that, you will find the Open-source Model description, like the one in picture zephyr 7B Beta. It gives a high-level notion about the capability of the model. Scroll further down, and you will find many other models. Now, let’s move ahead and search for a suitable model for us.
3️⃣ Model Search and Navigation 🔍
Users can browse and select from various models available on Hugging Face platform. LM Studio provides detailed information for each model, aiding in informed decision-making.
The next step for us is to choose a suitable LLM depending on your project’s specific needs. LM Studio’s support for a range of models offers the flexibility to select the most fitting option. If you click the search navigation on the left, you will head toward the Model Search area. LM Studio is great for the management of different model versions, including quantized versions tailored to the user’s system specifications. Arrows below mean the following:
Search Box: Model search. You click on search icon and head to search box. You can also head there from search box on Home Page. Find out what model would fit best for you and then navigate through it. I searched for Mistral, there are many other open-source models which you can also find in the home page.
Models: Navigation through Mistral models. You can choose one of the many options for Mistral and select a specific one. There are many options for that. You can see when the model was uploaded, how many upvotes and likes it has, what is the size of the model(in GBs), and how many parameters it has.
Filtering: You can filter out a specific model by time(recent), most likes and upvotes, downloads, and other filtration options.
Should work: One of the great features of LM Studio is that it tells you if the particular model will work on your workstation(laptop). Most of the low-parameter models worked on my M1 laptop. So anything up to 13B parameters worked fine.
4️⃣ Download the model
Once you select the model, you can download it and check out the status of the download. Before doing so, make sure that it will work on your machine. “Should work” in my case means, I can run this model locally. I looked in 70B Stable Beluga 2 option, and it said: “Requires 30GB + RAM”, which is not available in my machine. It is better to skip that then.
5️⃣ Interactive Chat Interface 🗨️
An integral feature of LM Studio is chat interface, which allows users to interact with the models. This interface includes adjustable settings for various model parameters, enhancing user control.
Chat Interface: Select the chat icon to head to the chat interface.
Start new chat: Click New Chat button to start the chatbot.
Downloaded models: Select one of the models, that you downloaded from previous steps. You will find them as shown in below pictures.
Configuration: You can play around with the configuration. You can select different Presets. In my case, it was Default LM Studio macOS. It performed the best for me.
In this particular chat, I asked the following question:
What do you think leadership will look like in 10 years from now, considering that AI, particularly LLMs will affect every part of our lives?
And Stable Beluga 13B from Stability AI had this prompt for me. I shortened it because of the size of text(in the comments section, find the full prompt 👇👇👇). I got a great reply.
leadership in 10 years will be characterized by a symbiotic relationship between humans and LLMs, with leaders who can effectively leverage these advancements to drive progress, innovation, and ethical decision-making being the most successful.
And it was swift, just using my laptop M1’s capacity. Incredible!!!
6️⃣ Developer Tools 🛠️
LM Studio offers a local server feature for developers looking to integrate LLMs into their applications. This feature is compatible with OpenAI’s API , facilitating easy integration.
7️⃣ Storage Management 🗄️
Users can manage the models on their system efficiently, adding or removing them as needed to manage storage space effectively. I can delete any model I don’t use. It is good to play around the models and then select the ones that is the most suitable in your case. Then delete all the rest that you won’t use that much.
Conclusion
For businesses and developers, LM Studio offers a strategic edge. It’s the secret sauce for rapid prototyping, testing, and deployment, all without the headaches of cloud-based delays and data privacy concerns. This advantage is invaluable in sectors where data sensitivity and quick turnaround are essential. From startups to enterprises, from healthcare to financial institutions, these kinds of tools will become a necessity in future development.
LM Studio emerges as a powerful and user-friendly tool for anyone interested in exploring and utilizing LLMs. With its user-friendly interface, it welcomes everyone — from curious individuals exploring AI for the first time to seasoned developers crafting the next big thing. LM Studio democratizes access to AI, enabling users to leverage the power of LLMs on local workstations.
LM Studio is not just about what you create but also about the fun and discovery along the way. Share your stories, tips, or epic fails in the comments. Let’s navigate this exciting AI wave together — one click, one code, one breakthrough at a time with LM Studio! 🏄♂️👩💻
Check out my video guide on LM Studio. Let me know your thoughts!