Uncensored LLM using a rented cloud GPU
Why?
When researching topics in cybersecurity, large language models like ChatGPT censor a lot.
So, we can host our own that is uncensored and it will help us learn things more efficiently.
You can rent a powerful $7,000 GPU at RunPod for $0.90 an hour. You can start and stop the machine on demand and only get charged for the time you are using it.
Go to https://www.runpod.io/ and click sign up. (NOT an affiliate link)
Add funds your account.
Click on “Go To GPU Cloud”.
I will select the RTX A6000, it has 48 GB of VRAM and is capable of running 70B Models. It costs $0.79 an hour in Feb 2024.
Click deploy.
In the upper right where it says “Type to search for a template”, click the drop down menu
Scroll down, select “RunPod TheBloke LLMs”, it should be running ubuntu 22.04.
Click continue.
Click connect
Select HTTP Service [port 7860] Note: make sure you select port 7860, my mouse icon is hovering over the wrong one in this screenshot.
Click the tab at the top that says “Model”.
I am going to be using the model called; Wizard Vicuna 30B Uncensored - GPTQ, because it is good at writing code. It is located here:
https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GPTQ
Click the copy button next to the name of the model to copy the model to your clipboard.
Paste in the model in the download section box where it says “Download model or LoRA”
Click download
The model will take a while to download. Dont click anything, just wait a while until it is done. It took around 2 minutes for mine to finish downloading.
Click the checkbox next to “Autoload the model”, and then click the refresh button. The refresh button looks like a blue recycling symbol.
Select the model from the dropdown menu.
Click Save settings to the right after you have the model selected in the drop down menu.
Click the Chat tab on the upper left.
You now have an uncensored LLM to ask questions. There are a lot of settings or other models and things you can do.
Note: Don’t forget to stop the machine when you are done, or else it will keep charging your balance.