Learn More
Resend Email
6
fasterthanlight
2d

I'm working on a project, and we ran into a problem where the server memory was getting used up way faster than before. At first, everything was fine because the server was restarting during releases, which cleared the memory. But over time, the situation got worse, and now it only runs for a few hours before crashing.
What can be done in this situation?

question

#bugfix

#memoryleak
Favorite
Ranter
Comments
  • 3
    2d
    We tried solving this by switching from active_model_serializers to blueprinter. Blueprinter was chosen because it uses less memory and is faster at turning data into JSON. This change immediately reduced the number of strings created and lowered the memory load.
  • 3
    2d
    @fasterthanlight Is this Flask?
  • 5
    1d
    Use a profiler and hunt down the memory leaks?
  • 0
    1d
    If your choice of json seraliser makes the difference between running out of memory on the server or not, then the thing you're serialising must be absolutely hideous.

    Whatever it is, you almost certainly don't need to serialise that bastard all in one go.
  • 2
    1d
    Sounds like go SAX or get (butt)SEX'd
  • 2
    1d
    Now for real. That's not usual behavior from trying to json.parse something too big.

    You have a memory leak somewhere. As @Lensflare said, profile.

    But it's most likely some unbounded cache behaving faulty.
  • 0
    6h
    py-spy or valgrind @Lensflare
devRant © 2021 Hexical Labs LLC
Privacy Policy  |  Terms of Service
Add Comment