The github page indicates it is a python project. You might get more results posting in that forum.
Rest assured, if @VinceL knows about malloc they'll know about swap as well.![;-)]( "Wink")
Modern operating systems (OS) require what is known as an MMU (memory management unit). This allows the OS to give more RAM (random access memory) than exists to each program. This happens through a mechanism commonly known as "faulting". Essentially when the OS runs out of RAM if hunts for apps which have been dormant for a while and steals their RAM by dumping into on a disk. It then gives some/all of that stolen RAM to the active app making the request for more RAM.
When an inactive app becomes active and tries to access its stolen RAM, there is a "fault". The OS then hunts for another inactive app, repeats the theft (dumping that to disk) and hands it back to the (in)active app - which is oblivious to the fact this ever happened. The place on disk where the OS stores the contents of this stolen RAM is known as the pagefile (aka swap). It is fixed in size. If enough apps grab (malloc) enough memory without handing it back (free) then the pagefile fills, the OS has run out of options and has to fail the memory request. It's at this point the app ought to fail gracefully although some may not bother checking if their request (malloc) failed and will likely crash.
I said the pagefile is fixed. It can be increased but there is a limit to how far this can sensibly go. You can end up in a situation where the OS is doing little more than dumping and restoring RAM from the pagefile. All the apps slow to a crawl so badly they can appear stopped. Some may even crash because they need to be responding to events but can't. This is known as "thrashing". When and how thrashing occurs depends on the circumstances. Consider it randomly unpredictable because ultimately is depends on what the apps happen to be attempting.
Thus, if there is an app which requires a large amount of memory, it may be possible to increase swap to allow that app to run. I have an app which for one period, requires many Gb of ram - more than the 4Gb that the rpi4 has. I nurse that though by setting a 16G swapfile on that rpi4. Once the app is past that point, everything is back to normal.
The other (and more common) is what @VinceL said: an app is asking for memory and never giving it back.
There is a command called 'top'. In fact there are many commands which can help figure this out but you need to understand the above to make sense of what they're telling you. Linux, for instance, will gobble up all the available RAM. You'll fire up 'top' on an idle freshly booted system and think there's no free ram. This is because having unused memory is pointless. Linux grabs it all for use as a disk cache. When an app makes a memory request, Linux makes its file cache smaller and hands some of that to the app. Also there's no point in the OS actually writing to the pagefile if it can keep it in the file cache: from a human perspective this makes no sense but it works. Added complications are the OS needs to periodically flush the file cache to disk (else you'd lose all your data on poweroff). Also applications are allowed to use other calls than malloc/free - ones which manipulate things (virtual memory) with more control. Google "virtual memory" and "dirty pages" if you want to go down the wabbit hole!
I don't know python. There may be a python debugging tool easier than 'top'. In any case it would be most useful if you can engineer a situation where the problem happens consistently and is repeatable. Ask on "pi presents" for logs. It may be the same "file/action" which is the cause. Consider: VLC on my rpi4 plays movies just fine except for two situations. First one happens so rarely I can't be bothered with it - I might occasionally have to reboot. The second is a function of the movie data: the solution there is to use a tool to transcode the janky movie into a sane format and delete the original.
Rest assured, if @VinceL knows about malloc they'll know about swap as well.
Modern operating systems (OS) require what is known as an MMU (memory management unit). This allows the OS to give more RAM (random access memory) than exists to each program. This happens through a mechanism commonly known as "faulting". Essentially when the OS runs out of RAM if hunts for apps which have been dormant for a while and steals their RAM by dumping into on a disk. It then gives some/all of that stolen RAM to the active app making the request for more RAM.
When an inactive app becomes active and tries to access its stolen RAM, there is a "fault". The OS then hunts for another inactive app, repeats the theft (dumping that to disk) and hands it back to the (in)active app - which is oblivious to the fact this ever happened. The place on disk where the OS stores the contents of this stolen RAM is known as the pagefile (aka swap). It is fixed in size. If enough apps grab (malloc) enough memory without handing it back (free) then the pagefile fills, the OS has run out of options and has to fail the memory request. It's at this point the app ought to fail gracefully although some may not bother checking if their request (malloc) failed and will likely crash.
I said the pagefile is fixed. It can be increased but there is a limit to how far this can sensibly go. You can end up in a situation where the OS is doing little more than dumping and restoring RAM from the pagefile. All the apps slow to a crawl so badly they can appear stopped. Some may even crash because they need to be responding to events but can't. This is known as "thrashing". When and how thrashing occurs depends on the circumstances. Consider it randomly unpredictable because ultimately is depends on what the apps happen to be attempting.
Thus, if there is an app which requires a large amount of memory, it may be possible to increase swap to allow that app to run. I have an app which for one period, requires many Gb of ram - more than the 4Gb that the rpi4 has. I nurse that though by setting a 16G swapfile on that rpi4. Once the app is past that point, everything is back to normal.
The other (and more common) is what @VinceL said: an app is asking for memory and never giving it back.
There is a command called 'top'. In fact there are many commands which can help figure this out but you need to understand the above to make sense of what they're telling you. Linux, for instance, will gobble up all the available RAM. You'll fire up 'top' on an idle freshly booted system and think there's no free ram. This is because having unused memory is pointless. Linux grabs it all for use as a disk cache. When an app makes a memory request, Linux makes its file cache smaller and hands some of that to the app. Also there's no point in the OS actually writing to the pagefile if it can keep it in the file cache: from a human perspective this makes no sense but it works. Added complications are the OS needs to periodically flush the file cache to disk (else you'd lose all your data on poweroff). Also applications are allowed to use other calls than malloc/free - ones which manipulate things (virtual memory) with more control. Google "virtual memory" and "dirty pages" if you want to go down the wabbit hole!
I don't know python. There may be a python debugging tool easier than 'top'. In any case it would be most useful if you can engineer a situation where the problem happens consistently and is repeatable. Ask on "pi presents" for logs. It may be the same "file/action" which is the cause. Consider: VLC on my rpi4 plays movies just fine except for two situations. First one happens so rarely I can't be bothered with it - I might occasionally have to reboot. The second is a function of the movie data: the solution there is to use a tool to transcode the janky movie into a sane format and delete the original.
Statistics: Posted by swampdog — Sat Mar 23, 2024 11:03 pm