Ranter
Join devRant
Do all the things like
++ or -- rants, post your own rants, comment on others' rants and build your customized dev avatar
Sign Up
Pipeless API

From the creators of devRant, Pipeless lets you power real-time personalized recommendations and activity feeds using a simple API
Learn More
Comments
-
AvatarOfKaine3745105d@retoor yeah except I'm only rubbing 5 to 10 threads on small html fragments I'm parsing.
And the Lan is fucking throttling me to like one 5 pages every 4 seconds using wget -
AvatarOfKaine3745105d@retoor nm I remembered the reason I'm using wget is requests was throwing an error
-
AvatarOfKaine3745105d@retoor the problem with using Asuncio and requests is the server or some part of the stack was rejecting it
-
AvatarOfKaine3745105d@retoor Exception has occurred: SSLError
HTTPSConnectionPool(host='frs-public.epa.gov', port=443): Max retries exceeded with url: /ords/frs_public2/national_kml.registry_html?p_registry_id=110001406194 (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1028)')))
ssl.SSLError: [SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1028)
During handling of the above exception, another exception occurred:
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='frs-public.epa.gov', port=443): Max retries exceeded with url: /ords/frs_public2/national_kml.registry_html?p_registry_id=110001406194 (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1028)'))) -
AvatarOfKaine3745105d@retoor
During handling of the above exception, another exception occurred:
File "/home/john/Documents/placeflattener_git/Utils/EPAScraping/htmltableextractor.py", line 56, in <module>
html = requests.get('https://ofmpub.epa.gov/frs_public2/...')
requests.exceptions.SSLError: HTTPSConnectionPool(host='frs-public.epa.gov', port=443): Max retries exceeded with url: /ords/frs_public2/national_kml.registry_html?p_registry_id=110001406194 (Caused by SSLError(SSLError(1, '[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (_ssl.c:1028)'))) thrown by request.get of a url -
AvatarOfKaine3745105d@retoor this is a hefty government server that serves these pieces of data to platforms like arcgis and Google earth and I'm simply getting error 4 from wget
once.. there was this code that
had major bottle neck
when performing get calls
and when i finally fixed it
the feds
called me and told me to stop it
they said it was cuz that
the client hit their site
soooo hard
mmmmmmmMMMMmmmMMmmMMMMMMMmmmMMM
and then there was this time that
I adjusted the hitting rate
to something a little slower
and when I finally ran it
dem feds
they didn't even notice
I couldn't quite explain it
maybe if I raised the limit
hiiighhhh errrrrrr
MmmmmmmMMMMMMMmmmmmMMMMMMmmMMM
hahaha crash test dummies :P
rant