In the last 24 hours, bird.makeup pulled 127Gb of tweets (just the text and metadata and no media) from Twitter
🤯 🤯 🤯
@smallsco I think that's a good idea. There a lot of magic numbers like that I should move to env vars.
@smallsco You can now tweak that with: Instance__ParallelTwitterRequests=2 or whatever
@vincent Thank you! I'll give this a try later today.
@vincent Dropped it down to 3 and I'm not seeing any 429 errors from the Twitter API any more, thanks!
@vincent Holy shit. That's nuts.
@vincent What do you think about exposing the amount of parallelism as an environment variable? I've been getting a lot of these errors in my instance logs since the last update:
error.message:Response status code does not indicate success: 429 (Too Many Requests).