Why is it that if you are not able to remember every single detail about something you've heard, that you as a person are deemed worthless?

Everything you do and need in life is bound to your memory.
Can't remember a name?
That's rude.
Can't remember what a technology does?
You won't get that job.
Can't remember the topics you learned about in school?
You won't get that education.

I can't remember things, my mind is constantly drifting. This, together with my inability to articulate myself clearly, makes me a complete nobody.
I hope that someday I'm just able to do something creative and not have these issues. Until then I'll just try not to jump in front of a train.