Multi-platform desktop app to download and run Large Language Models(LLM) locally in your computer
๐ Download ย | ย Give it a Star โญ | ย Share it on Twitter ๐ฆ
Features
- ๐ The power of AI in your computer
- ๐ป Local - it can work without internet
- ๐ Privacy first - your messages don't leave your computer
- ๐คซ Uncensored - you can talk whatever you want
- ๐ Open source
Try the app
If you just want to get the app installers and try the app, go to secondbrain.sh.
How to use
The first time you open the app you will need to download a model, and then activate it.
Download a model
Secondbrain comes with some models ready to download, that we know work. You can check, or modify, the models.json to see their details.
You can also add your own model files to the /models folder, and then activate them from within Secondbrain app. The model needs to be in ggml format.
Activate the model
Just select the model and press "Activate model", and you are ready to start using the model.
The prompt is important
Language models are predictive machines, you throw some words(tokens actually) at them and they try to predict what is the most likely token to come after that, and after that new one, and so on. Not all the models work so smooth as ChatGPT, it depends on the pre-training, the fine-tuning, and the under-the-hood prompting.
When using models you need to take into account what format they understand better, for example Alpaca models were trained with this format:
Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response:
So, if you want to download and use your own models, take into account the prompt and change it in the configuration screen.
With foundational models, like Llama, things get crazy, there is no fine-tuning, so finally you can flex your prompt engineering skills and play around.
How to run from source
The app is build with Tauri, so basically you need to follow this guide: https://tauri.app/v1/guides/getting-started/prerequisites/
Techstack
Contribution
Yes, please. Just send your PRs.
Gracias ๐ค
Updates
For updates follow here: @julioandresdev