Did you know that docker's ADD instruction uses "go-http-client/1.1" as user-agent when src is an URL?

I didn't. And since I'm unfortunate, enough so that this user-agent is blocked by my company, I've now spent twice the time it took me to write the whole dockerfile to identify the problem and fix it...

I love waisting my time for such minor things...

Aren't agents easily spoofable? What kind of security measure is that?

@alexbrooklyn I needed ADD for busting a cached git clone. Theoretically, it's nice and simple.
But ADD does not allow you to change the user-agent (at least in my knowledge).

It's easily fixable by downloading the wanted files with an external script and then use ADD to add those files in the container, but it's not as nice and simple...

Without mentioning the awkward moment when you find that ADD does not work but wget has no problem whatsoever

@Edlundin I think the comment was to the firewall blocking, not the docker add.

@Voxera Correct, I didn't pay enough attention

@alexbrooklyn To respond to your question about security, I simply don't know. Some attempt to mitigate bots/crawlers I suppose? But the first thing I would do if I was to write a bot is to change its user-agent so...

User Agent blocking is a shady thing...

In general, relying on or parsing the User Agent is questionable.

Many do it for the sake of simplicity, though you could just grab a list online and let e.g. a python script test 1000s of user agents of the list one after another.

It's highly likely that you'll find one that gets through.

Rest is setting up an haproxy or nginx as a proxy for the docker which replaces the user agent header on the fly.

As the docker registry is just a regular HTTP service, hosts file DNS highjacking and et voila.

Protection is bypassed.

(I might have done this a few times.... Don't ask.)

Edit: or in this case, proxy the site. It's trivial.

Nice find of a silly implementation!

Could (or have) you filed an issue with them to allow changing it through configuration?

@IntrusionCM @alexbrooklyn it's probably not to protect against attacks but automated scanning. Automated scanning is usually not targeted and thus this is a valid measure against it.

@AlgoRythm automated scanning is an attack form....

@IntrusionCM sure. The main idea of my comment was to point out it's to protect against non-targeted attacks, which it is effective against.

@Flygger Following your suggestion, I just did!

Wonderful @Edlundin! Then the world got slightly better today too :D