So I spent the whole day optimizing bird.makeup and got about a 5x in performance. Things are starting to run better, seems we are iterating on all accounts within 4-5 hours right now. I have other ideas that will probably get another 3x that I want to implement tonight, so performance should be right on target.
Until more people sign up of course! ๐
@vincent What are the system requirements, if any, for running your own instance?
My cloud server has 2 GB of RAM and it keeps running out of memory. I had to add a memory limit to the birdmakeup container in the docker-compose.yml file to keep the system stable (but now the container is killed/restarted every half hour or so).
When it works, performance is great though ๐โ Thanks so much for building this!
@smallsco Yeah, there is a memory leak somewhere... I'm hunting for it!
I just took a quick look at it (by the way: nice code). Three things potential things I found (no guarantees):
1. The used version of Newtonsoft has a potential memory leak: https://github.com/advisories/GHSA-5crp-9r3c-p9vr
2. MagicKey has a property `_rsa` which needs to be disposed. (there might be more)
3. Quite a few regex that might never complete.
I would need to actually run the application for my tools to find something, but maybe I'm lucky with the above.
@DevWouter @smallsco Thanks for the compliment! But the credit goes to @BirdsiteLIVE .
I think I figured it out. I took a memory dump, and noticed large allocation related to SQL queries. When I removed Dapper and used vanilla npgsql the problem when away. It's really not obvious to me what was happening there, but it might be related to your number 1
@smallsco No problem! I just did the most frequent queries, so it's still leaking, just way more slowly
@vincent With the latest version, the CPU and network usage have gone up slightly, but there's been no change whatsoever that I'm seeing to the memory usage ๐โ
I updated the docker-compose.yml file to enforce a 1.5 GB memory limit (my server has 2 GB), which is what causes the consistent drops you see in the attached graphs:
-----
deploy:
resources:
limits:
memory: 1500M
-----