Honest question. When do you consider yourself a "Big data engineer"?

Today I managed to create a system that collects historical metrics from monitoring tools every 5 minutes and do all sorts of crazy transformations to make them ingestible by grafana Mimir in OTLP protocol. Doing 600gb a dat, millions of active time series, .... And I still feel it's, "small"

Thoughts?

My data is bigger than yours.

@electrineer are you a big data engineer?

You're doing transformations on gigabytes of data. That's big.

Hamilton. That's the only case I think that someone working with code is justified in calling themselves an engineer. Doesn't matter if it is data or software or whatever.

And I am getting very suspicious about the claim that someone is an engineer when they have never visited a materials class.

NeatNerdPrime, if your real first name is not Margarete, I don't consider you an engineer.

I've seen anecdotally that it's not big until it's terabyte scale. But you're pretty close.