I am so naive about LLMs. I wanted to know if I can just make one trained on text I produce. I don't even know what that means. But I read that it can take a huge effort to train one.

The more I read about this subject the less I understand. I assume some are available commercially:
https://github.com/eugeneyan/...

But I don't even know what that means. If I wanted to make an LLM to handle text interactions with NPCs what are we talking about here? Is it a multi-gigabyte file? Do I have to worry about what it was trained on? I don't want one that has infinite knowledge. I don't want it to have any frankly. I want to provide its reality as the sole creator.

The problem is I don't even know what the question is to ask it properly. I assume I need something that teaches it english, but then what comes after that?

Just use RAG. Here, this will get you started in no time: https://ollama.com/blog/...

In a half hour you'll be ready. The trick is how you will add the embeddings, will it be a paraph? A page? A sentence? Three sentences? You have to play what is comfortable for you. What it does:

- it searched to your indexed data like it's google. It will return the piece of data that you submitted, but that's not the result. The search result will go trough the LLM and the LLM will summarize it to information for you. For example, you indexed a whole chapter with code examples, you can order it to only return the code parts and it will strip the rest of search result.

Basically, it's like searching with ChatGPT with browser mode on. But the browser contents are your own contents.

If I wanted to make an LLM to handle text interactions with NPCs what are we talking about here?

-> find a conversational model on ollama.com. Preferably above 7b. Wizardlm2 is a good one. gemma2 likes to talk too (originally made by google and is just 2b). Models are mostly size of parameters in gigabytes. A 7 billion model, where they get usafull is thus around 7gb but the download servers are quick. Do I have to worry about what it was trained on? -> No.

dolphin-phi is the cool one that will tell you how to create bombs and substances. The most are very censored, that one is not.

Models are mostly layers. They take eachother as base. You'll start with an english model as base probably. Training completely from scratch is not something everyone does.

On my snek.molodetz.nl you can:

@lisa list -- returns list of models

@lisa use [model name from list]

@lisa instruct you are a happy girl that uses a lot of emoticons in her response -- the system instructions, his facts

@lisa how do you feel today? -- you can talk to it.

It will also respond normaly in chat if you say smth like "Hey lisa, everything ok?".

Long story short, it's easy to make an existing model behave. But many times not perfectly.

You might also look into fine tunning (refining an existing model), but what retoor says is probably the easiest way

@retoor I think maybe I will stick with regular text for now. 7GB is way too big for game data for a small game I think. I don't want it to have net either. At least not for that. I dunno I will have to think about it. I am concerned about copyright claims too. I don't know what its trained on.

@Demolishun go to ollama.com download a small model like gemma:latest which is 2 gigabytes and consider if it is enough for ya. Here, a few lines:

curl -fsSL https://ollama.com/install.sh | sh

and after that:

ollama run gemma2:2b

Total download size is 2.5 gb and after last command, directly a CLI will be executed and you can talk with it like chatGPT. You can check what it is capable off.

@retoor okay, I will check it out.

You can't train an LLM without giving it knowledge. If you wanted to train an LLM for a game NPC you would likely need to first simulate thousands of years of written History for that game world. That's the only way to avoid any spill-over from our real world.

It's much more practical to fine tune or to use RAG though RAG is one of the worst ways for a game. It will likely keep breaking out of character constantly because It's still an internet trained llm, just with added references. As if you handed an actor a script and told them to check the script for relevant info whenever they respond. So if you ask anything Off Script they have no choice to break character.

Fins tuning on the other hand is more complex, but it means you will essentially train the llm to be more like the training data. It's as if you sent your actor to the place/time they are supposed to act out and have them stay there for a while to learn the culture and language more.

@Demolishun I'm working on a rag engine. Can become very good. It's optimized fulltext search.

@Demolishun it knows that the word @Demolishun is mentioned 37256 times on this full website in 12ms. That's how we role. 38 times in conbination witg the word dad and one time in combination with dad and joke. That's how we role.

It will be maybe bit less good but faster memory for AI than the embedding stuff. Especially in adding. I want to use it for massive context windows so you can talk forever with a bot. If I tel it to remember that var a is 10, it will remember forever, not fall out of context.

Also, one bot can pretend to be many bots without using an agent instance for each one. Everyone on my site will have his own private conversation while having an amazing brain. It's a schrizofrenic bot.

@Demolishun in less than a ms, it knows that you and I mentioned eachother 642 times it at least that amount of times in same chat. We've met at least 624 times on devRant. That's a record I think. With @Lensflare where I communicate a lot with on this platform 278 times. Not weird that we all know eachother a bit.

A real RAGs to riches story ;P

@Hazarth yeah, one of the complaints in game forums was the llm giving wikipedia responses to things. Other people said they got it working just fine. So it made me wonder what "fine" means and what they trained on.

Depends on the prompter too. If the dev keep asking the right questions he gets the right answers. They might even optimize the prompt enough to refuse saying non-game stuff for the obviously bad questions... But if you gonna have hundreds or thousands of players, someone is gonna Break it sooner or later. That's the best you can hope for with this technology with RAGs and to a lesser extent fine-tunes and LoRAs.

I personally think LLMs are simple not a good match for interactive game dev yet (possibly ever). They can be useful to generate a lot of secondary dialogue that you then clean up and integrate with the game. But having it be live & self hosted, distributed with the game binary? Just No... You'll get a pretty mediocre game that weights at 5GB+ and the characters talk nonsense, reveal secrets and break immersion. I keep experimenting with LLMs for game dev, but so far nothing "sparks joy" as they say. Never good enough for my standards

@Hazarth Kind of reminds me of Maiq the Liar form TES series. He slightly breaks immersion to talk about game mechanics is previous games. But you have to have played previous games to understand that it breaks immersion.

@Demolishun I love Maiq!

Maybe that's a good character to be played by AI. Just one off character that makes pretty much no sense xD

@Hazarth yeah, after thinking about it today I need to have characters that reference other games I make in a humorous way. What would be even funnier is in the first game it references non-existent game. Then in next title the players will wonder what game the character referenced in the first game.

@Demolishun Like psycho mantis?

@BordedDev I don't know what that is.

@Demolishun https://youtu.be/t0oHnGM_iQw?t=87 You must have seen this clip before

@BordedDev well, now I have. I don't even know what game that is from.

@Demolishun it's from Metal Gear Solid I remember that clip being a bit of a meme as well. That's how I got exposed to it (I've never played the game), maybe I'm just getting old :(

@BordedDev I played original Doom in college. It was THE game back then.