once.. there was this code that
had major bottle neck
when performing get calls

and when i finally fixed it
the feds
called me and told me to stop it

they said it was cuz that
the client hit their site
soooo hard

mmmmmmmMMMMmmmMMmmMMMMMMMmmmMMM

and then there was this time that
I adjusted the hitting rate
to something a little slower

and when I finally ran it
dem feds
they didn't even notice

I couldn't quite explain it
maybe if I raised the limit
hiiighhhh errrrrrr

MmmmmmmMMMMMMMmmmmmMMMMMMmmMMM

hahaha crash test dummies :P

@retoor yeah except I'm only rubbing 5 to 10 threads on small html fragments I'm parsing.

And the Lan is fucking throttling me to like one 5 pages every 4 seconds using wget

@retoor nm I remembered the reason I'm using wget is requests was throwing an error

@retoor I'm running the process's async

@retoor the problem with using Asuncio and requests is the server or some part of the stack was rejecting it

@retoor yes and I'm trying to remember what the error message was

@retoor you do remember this conversation right ?

You alive yet ?

@retoor Exception has occurred: SSLError
HTTPSConnectionPool(host='frs-public.epa.gov', port=443): Max retries exceeded with url: /ords/frs_public2/national_kml.registry_html?p_registry_id=110001406194 (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1028)')))
ssl.SSLError: [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1028)

During handling of the above exception, another exception occurred:

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='frs-public.epa.gov', port=443): Max retries exceeded with url: /ords/frs_public2/national_kml.registry_html?p_registry_id=110001406194 (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1028)')))

@retoor
During handling of the above exception, another exception occurred:

File "/home/john/Documents/placeflattener_git/Utils/EPAScraping/htmltableextractor.py", line 56, in <module>
html = requests.get('https://ofmpub.epa.gov/frs_public2/...')
requests.exceptions.SSLError: HTTPSConnectionPool(host='frs-public.epa.gov', port=443): Max retries exceeded with url: /ords/frs_public2/national_kml.registry_html?p_registry_id=110001406194 (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1028)'))) thrown by request.get of a url

@retoor so it says right there but turning it back on didn't work

@retoor however the slowness is obviously a lan thing

@retoor this is a hefty government server that serves these pieces of data to platforms like arcgis and Google earth and I'm simply getting error 4 from wget