There are a few important terms to learn and understand with AICom. That will be helpful when communication issues you have AICom for help. But also to understand what you can do with AICom, it helps to know these terms!
Integrations
Connections to 3rd party services and API's for IO purposes.
(To the user subsystem are services, but maybe they need to be renamed subsystems)
Examples:
Actions are functions that execute through commands. These are ways the AI interact with features of
AICom, including integrations.
Examples:
Commands are a collection of actions. These are the triggers for your AI that execute the collection of actions. These are configurable in how they are triggered.
ACS / Abstract Command SystemThis is the system that parses text heard for trigger phrases, then binds them to commands, which can have any series of actions that AICom executes.
TriggersThese are ways that commands can be configured to execute. Such as:
similar to engine, but defines a collection of components and subsystems that aim to achieve a goal.
Examples:
A provider for a component or subsystem:
Examples:
A driving component in a system that completes a major function:
Examples:
A means of connecting a sub system to a service(s) Each subsystem has adapters to each of it's various services.
IO
Components that are specifically used by subsystems for the purpose of the bots IO
Examples:
Puppeting refers to the way of controlling your AI over OSC or other interfaces. This distinction is important because the AI can do a lot of things automatically, but a few things still need human intervention. This includes moving the AI around, interacting with objects, and even in rare cases, making it speak for you. This is all referred to as "Puppeting" your AI.
nVidia based GPU with 6GB of dedicated VRAM (Not shared)
32GB System Memory
Modern 4 core processor or better
~50GB hard disk space
You will want more than enough hard diskspace for experimenting with various models.
Since AICom and AICore let you use any number of models, it's important to understand that this will vary widely depend on how AICom is used.
VRAM usageAI models are usually run on the GPU. Models will use a varying amount of VRAM based on a few different factors. For example, let's discuss: "TheBloke/Llama-2-7B-GPTQ". The key components of this model are 7B and GPTQ. This is how it breaks down:
https://www.youtube.com/watch?v=ceMbjP11zT0
Full Walkthrough of AI Jokenhttps://www.youtube.com/live/-6URSeKkyuY
Original Documention for building AICore and GUI
Installation and Setup Guide
This guide covers build requirements and other information that may be useful for users running AICore from python source.
This usually gets installed with a fresh windows installation, or with Steam, and many other applications.
But just in case, here it is:
https://aka.ms/vs/17/release/vc_redist.x64.exe
If you want to use Coqui TTS as your backend for AICore, you will want to install ESpeak-NG for this.
https://github.com/espeak-ng/espeak-ng/releases
AICom uses SQLite for it's configuration database.
We recommend using SQLite Studio for accessing and editing information in the database externally.
Likewise, both AICom and AICore have their own API's that can be accessed.
For this we recommend Post Man for testing the APIs.
This can be found at:
%localappdata%low\StudioJAW\AIComAnd contains a number of useful things. Performing a Factory Reset
Before deleting everything in this folder, consider only deleting AICom.sqlite. Deleting everything in this folder will factory reset AICom, but it will also delete any user files you may want to keep.
If you just want to reset your configuration, then deleting AICom.sqlite is a better option.
The LogPlayer.Log is also located in the main data folder, which contains useful information that could be neccessary for determining problems. If you need help, we may need to ask you for this. And if you do have questions, please feel free to ask in our Discord
The Model Cache FolderFollowing, there is the folder where Huggingface stores things:
%userprofile%\.cache\huggingfaceThis houses all models downloaded, and if you end up with a corrupt model you can delete it here to redownload. The AICore log file
<AICore folder>\aicore.log
These two numbers help filter or increase the randomness of the output of your GPT model.
Top-k:You will need to start by visiting the Discord Developer Portal. From there you will need to create a 'bot' for your, uh, AICom Bot. And remember to configure your intents.
For example, this is Joken's Configuration:
Further down the page is the Bot Permissions generator. Remember to generate a permissions Integer to use with your bot. I trust my own bot, so I give it 8 (Administrator), since it is not a public bot. If you should chose to make your bot public, remember to give it appropriate permissions.
Modify the following line with information for your bot:
https://discord.com/oauth2/authorize?client_id=<Your Bot ID>&permissions=<Your bot permissions to grant (Permissions Integer)>&scope=bot
Puppeting refer
User and AI Avatar Examples
It's pretty involved. Things are also going to be changing with the new command system in progress; however, I can at least release an example project to show how your avatar and your AI's avatar should be setup.
There should be two scenes with avatar to upload.
AICom avatar templates
Unity Package
Also, I have uploaded both of them to Joken's account and are public.
You can use both for testing purposes as well!
AICom Player Test Avatar
AICom Bot Test Avatar
The are available on pedestals in this world
Another approach to moving your avatar, is manually with OSC Link
OSC Link is the name of the system that allows you to control your AI from your own avatar. At current, this is designed mainly for VRChat, but could potentially work for other VR Platforms.
These instructions are also useful if you wish to run VRChat twice for both you and the AI on a single machine.
OSC link requires two seperately running instances of VRChat
In this example config, you will see that "Your Client" refers to the VRChat instance that you will be running in VR for yourself. By default you don't need to launch VRChat in any special way. You just simply start VRChat for this connection; However, you need to launch VRChat with special launch options. There are a few ways of doing this. You will want to use various command line options for VRChat, which You can learn more about VRChat's launch options here.
The .bat file approach
To create the .bat file, navigate to the Steam folder for VRChat (Right click VRChat in steam, select manage and then Brows local files).
Then in Windows Explorer, create a new text file and make sure the file has the .bat extension.
Open the bat file in a text editor and add the line:
start_protected_game.exe --osc=9002:127.0.0.1:9003 --no-vr --profile=1
Where "9002" and "9003" match the example configuration for your bot's client. "--no-vr" is added, in this case to prevent the 2nd client
from using VR, when you are running on the same machine and using VR for yourself. You will also notice this is "start_protected_game.exe"
and not "VRChat.exe", as it is required to start VRChat with EAC or you will get an EAC error. "--profile=1" tells VRChat to use an
alternate profile for your
If it is failing to launch, the most likely reason is because the file was not named with the correct extension. By default windows hides extensions, Here is a guide on how to show extensions in windows. Most like you may have created a file as ".bat.txt". To correct this, save it apropriately with notepad by selecting "save-as", and changing the "Save as type:" to "All Files (*.*)" and make sure the file name ends with ".bat".
The Creator Companion ApproachYou can also use the VRChat creator companion to launch VRChat with these parameters. More on that can be found here.
Copying the VRChat directory
You may want to copy the VRChat directory and all of it's content to another location, if you plan to run VRChat for your AI and yourself on the same machine.
The reason for this is so windows sound system can differentiate which copy of VRChat you are configuring for audio.
Oh, sure thing. Sometimes I run things in Linux myself, and this is the script I use when launching a 2nd instance of VRChat on linux:
export STEAM_COMPAT_DATA_PATH=~/compatdata
export STEAM_COMPAT_CLIENT_INSTALL_PATH=~/.local/share/stream/steamapps/common
~/.local/share/Steam/steamapps/common/Proton\ -\ Experimental/proton waitforexitandrun start_protected_game.exe --profile=1 --no-vr --enable-udon-debug-logging --osc=9002:localhost:9003
There exist a number of RVC models that exist on the internet. Please check the license and copyright of the model before using models for commercial purposes.
However; RVC in AICom is pretty straight forward if you already have the models. First thing to know is that it's an filter effect, so you need to select a base voice to start with (Usually a MSTTS voice). Then you just select the model path and index files for AICom to use, and it takes care of it. However the daunting part is obtaining a voice.
Creating your own
Here is a video on how you can clone any voice to create an RVC voice using google collab:
AI Voice Cloning for Singing with RVC
There is also a number of other videos in that playlist that may also prove useful.
Here is a video on how to clone any voice, assuming you have consent to do so, using RVC
to train it on your own machine:
RVC: Local Training Tutorial
I created Joken's from eleven labs originally. From there I used tools to pitch up and down the voice for singing. I then blended the voice with my own in the data set to train her RVC voice.
Making your AI SingThis is not an automatic process. There is a song's folder that exists for AICom to place precreated mp3's to have your AICom virtually sing. But to actually create the song, we recommend Replay Which is what I've been using to make songs for Joken and Delphi. Just like with AICom, you provide it your RVC model and a song, and it takes care of things for you!
You will need at least 1 virtual audio device for AICom to function. There are a few different options, but the AICom community usually suggests VB-Audio's virtual cable or voice meter (Same company). Typically you will want to set the devices you plan to use for VRChat as DEFAULT. As VRChat will want to use the default audio output device. You can over ride this as shown in the example for a single machine setup. But the basic concept of this is, you will need to configure the in of AICom to the out of VRChat and vice versa as explained in this diagram:
Single Machine setup To configure it for a single machine, you will have to use the Windows Sound control panel. You may want to also consider copying the VRChat folder and it's contents, so windows can differentiante between the two. This gets confusing as both will appear in the Windows mixer as VRChat, and you will unfortunately have to guess which one is the correct one.
Typically that setup for a single machine would look like this:
Unfortunately we don't have a better way to explain this, but if you need help with this, please do not be afraid to ask in our Discord server.
Linux
Linux has a built in way to enable a "Loop Back" device for all distros.
But specifically with Ubuntu based distros, you will want to edit or create the file:
/etc/modprobe.d/loopback.conf
Then add in it:
options snd-aloop enable=1,1, index=10,11
Which will create the 2 devices you need for AICom and VRChat. You can have more than 2 devices, just simply add more 1's to the enable option, and give each an appropriate index following 10,11. e.g. The next will be 12, then 13. It just has to be an unused index value.
If you are using pipewire/pulse audio (Standard to Debian based OS's)
AICom has an API for sending commands other than verbally or through discord. This is not to be mistaken with the AICore API. It's pretty basic and uses http (not secured). But it can be used for twitch integration and other applications to interact with your AI! You can enable and configure this in the Integrations tab.
The IP Address is the IP of the machine AICom is located on (The one you specified in AICom to listen on). The Port is also the port you specified. No special headers should be required. Some commands may not require a value to be entered, leaving this blank or omitting it should be fine.
You can send commands to the API in two ways:
[POST]
If using POST you can pass the command as a route, following the / in the url. The command value (Such as the search term for youtube) to "value", and the username of the person talking to "username" can passed into the post using a JSON object string. Please not that this is using a raw string body for POST operations.
[GET]
If using GET you can pass the command into "command", the command value (Such as the search term for youtube) to "value", and the username of the person talking to "username".
No special headers should be required.
The AICore documention and API can be found at:
AI Core API and Documention
Some like to run AICore from source, and some prefer to even keep it in a Linux environment while still using windows for
AICom and VRChat, as features remain more up to date in this environment.
If you should chose to do so, remember to follow this guide in setting up Cuda for WSL:
