Question - am I sending too many concurrent requests?

[jamietanna]

I've noticed that every so often - particularly when I was testing with a much higher concurrent request count (~250 goroutines all sending off requests) - I would receive HTTP 500s from Ecosystems.

What's the recommendation for how many concurrent requests I should be trying to keep to?

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[andrew]

Yeah you are likely DOSing the service! We don’t have a lot of funding for infrastructure so there’s limits to what we can provide right now.

There may be better ways to query the API based on the shape of the data you are trying to extract.

[jamietanna]

Sorry for the delay - thank you, yeah I'm gonna drop down the rate I hit Ecosystems, as well as make it so there's a little more caching in place locally to try and avoid a lot of unnecessary requests 🤞