@retoor I'm trying to get it working locally so that I don't have to send my or a client's stuff to a proprietary LLM in case they don't trust them or want to fine tune on their own data.

I'm testing with a 15k story I wrote so that I know for sure where it gets things wrong. The chunking strategy I have works decently even with 7b and lower models, with larger models giving more precise summaries. It stays fast and accurate this way, where sending the entire document at once leads to errors and hallucinations.

I've tested the chunking strategy on huge open models like OSS 120B, and one more that was even larger that I can't recall, on runpod, and the results aren't so much better than using a ~30B model that I can justify the cost, so I'm sticking with local.

I'm really hoping that using images can extend the context for general use. For non-reasoning models, this seems to be a promising experiment.

@Lensflare Reasoning vision models like Magistral read the entire document into the context, so it fills up pretty quickly, so I don't know how well it performs over several pages compared to non-reasoning models.

@Lensflare I've got a python script that spits out the pages as pngs with the options of merging pages. It seems to work best with one page at a time and seems to work better than processing an equivalent amount of text. It seems to be easier on the context window as well.

I have a 32GB 5090. I've tried Gemma 3 27b, Mistral Small 3.2 24B, Qwen VL 30b, and they all seem to like processing images better than the text. I think it's because they scan the image each time instead of filling the context with the text, so it has more room to work with.

I put the entire thing into the Grok API and it does better, but it used up more than 3 million tokens in about 2 hours of testing, so that's not very scalable.

I am really looking forward to some unknown person becoming a trillionaire when they figure out non-proprietary AI processing hardware.

Publicly available models are lobotomized to he point that they couldn't harm a fly, or count Rs in a word. And abliteration effectively ruins the effectiveness of any model it is done to.

The best you can hope for is roundabout trickery to coax it into saying stuff it's not allowed to, but more and more the censorship is baked into the training data, so models in the future won't even know things it's not allowed to say as opposed to being trained to avoid the topic.

The ones we have to worry about are the ones that are used by the government and the wealthy to control the world even more effectively, and there is zero way that they would adhere to any restrictions.

I love when a designer gives me a hard time about design issues that I can't even see until they're pointed out.

For one, once it leaves my world, it's up to them to make it look like they want so I have no idea what it's even supposed to look like.

For two, I'm not a designer, so I probably couldn't even figure it out if I knew what it was supposed to look like. Give me a screenshot and either tell me what the problems are or else it will never get fixed.

I'm not here to read your mind.

@wojtek322 Semantic tags are arbitrary and reeks of makework nonsense inserted by non-developers.

The heat is not even the thing you need to worry about. It's the gravity that rips apart the matter that then becomes the heat.

You have to understand. They're a small open source project that is strapped for resources. Open is even in their name for goodness sakes.

@Lensflare It's not even that AI autocomplete is bad. It's that the traditional autocomplete that used to be great is now always terrible because they want to push you to AI autocomplete that is a coin flip on usefulness. I'm just so sick of everything getting worse.

Thanks to forced AI in everything, autocomplete is basically broken, so all I use my ide for now is formatting and quick navigation between functions and making git and DB stuff easier.

@BordedDev In the very old United States they did but that went away with time. There are still parts of the world like that where they make you sleep at your post or rent you a cot and you don't really get to go home. Our corporate overlords are doing everything they can to spread that to the United States and Europe again because it's far more economically friendly to do it that way.

If it makes you feel better, you'd never be allowed to go home if it was legal for them to chain you to your desk.

I like that the chart updates in realtime.

@12bitfloat I use Debian with Mate for my primary development OS and plain Debian on all my servers. I really love how intuitive it is.

The downvoter is probably a bot or 4chan troll.

@jestdotty I'm so happy I'm solo in my company.

How would that work in multiplayer? I'd love to see time based mechanics in a PVP game.

@YourMom Yes and that's why everything is falling apart. It's all about how your coworkers make you feel and not what they are capable of.

Johnny was able to put together an MVP in a week but he wasn't very good culture fit so we had to let him go.

@retoor So will replacing the melting power adapter for my 5090. I can't shut the case anymore, but the random restarts have stopped.

@retoor I have mine set to reply in a concise way with no filler, so maybe that's why I'm not seeing that. I loathe cutesy crap like emojis coming out of anything I pay for to get work done.

I'm convinced that they add that filler to make their service cost more so you'll have to pay for more responses. Buy screw the sea turtles and the birds, so long as they get theirs I guess.

@retoor There are a minimum of three filters in my case. Not that I explicitly looked for one like that, but when I built this computer, I went a little crazy so it turns out I got a good case too. They are usually pretty dirty, but they are easy to get to so I clean them regularly.

No model that has been lobotomized by the silicon valley priest class will ever come close to anything approaching intelligent, let alone sentient.

@YourMom No other industry is allowed to sell a product then arbitrarily take it away from the user without warning or compensation.

If you want to sell a service that you can unplug at any point, you must properly market it as a service and warn people that they are gambling that their money wasn't just put into something you plan to unplug in 16 months.

On top of the consumer protection angle, there is a lot to be said about the negative cultural impact of games and other types of media just disappearing into the ether at the whims of people that can only see one quarter in front of them.

It's high time the games and software industries be treated like every other industry.

The corporations that spew hundreds of years of one person's pollution per day per vehicle are the ones at fault. Not the average Joe going to the grocery store or work.

The average Joe is also not the one responsible for the entire planet being saturated with forever chemicals and microplastics. They are also not responsible for the entire countries made unlivable due to nuclear and chemical contamination.

They are also not responsible for decades of pollution spewed into the air by volcanoes and naturally occurring wildfires.

But God I hate the sound of engines of any kind ruining my peaceful days, so I'm all for abolishing them as soon as possible.

There is going to be a massive overcorrection of this kind of thing in the next decade. We are seeing the beginnings of it now.

People that just want to live their lives, (read: the vast majority of people) are fed up to the point of taking action, which does not bode well for the ones that have steered society into the dysfunctional mockery that it is now.

Jobs being easy to lose and hard to find were the only thing keeping otherwise docile people from acting.

The best thing you can do in the coming years is blend in with the crowd and not try to force whatever your thing is on people.

@CaptainRant I'm not sure. But the filesystem has also started switching to readonly at random and my audio crackles when I start it up. It's getting real annoying. If wsl worked, I'd be golden.

@LillianQuinn Are you a real person or a spambot? No one having heard of whatever that is might be one of the reasons.

@antigermanist There are people not living lavishly that also don't die in the streets, so I fail to see how that matters. You need some vitamin d or something.

@jestdotty lol no. I keep that in my 4k square foot Morton building as printed hard copies.

One drive is full of LLM/SD models. One full of VM images. Two are full of pictures and movies and videos. One is an archive of old computers and hoarded assorted data. One is the system drive.

@tosensei Virtualbox. HyperV's performance is awful. I tried WSL2, but I couldn't swear hard enough at it to get phpfpm to work.

@rootshell Oh boy trying to get that going originally was a huge headache. For some reason, during the install it would reset the nginx service file which would show the placeholder site for new installs instead of the sites I expected to run when the service restarted. That cost me a couple days.

I imagine he had his hammers and wrenches modified to accommodate his refusal to learn how to use them as well.