Ugh... I don't like how TCP is a stream protocol and how UDP is unreliable and unordered.

I want a semi-reliable, ordered, message protocol dang it!

I think the problem is ordered. Once you need that you more or less need reliable and if you skip that you still need the same extra checks on your side that udp needs.

@Voxera yup. But it's mostly a comment on the fact that I send packets with my higher level protocol.

With UDP that's trivial. Just send a packet!

But with TCP, I need to encode it into a stream and then later pull it out again. That's just a waste of efficiency and lines of code.

You can build that on top of UDP easily, i.e. some Xmodem style ACK/NAK protocol. Obviously, network latency will kill your throughput in this case, which is why nobody does that.

You may want to look at 0MQ I guess?

@DirtEffect I know about it, but it's a bit too opinionated and I can't exactly do what I want with it.

Also, I'm in embedded land, so not everything is very supported.

@Geoxion Ah I understand... Good luck with your search then!

@Geoxion I’ve come across a few instances where my team has made “smart” UDP but adding logic to both sender and receiver that allows for ordering each piece of data and then sending back a message that tells which pieces have been sent and received.

@jeeper I know about that and I'd like something like that. It's usually done in low latency applications like games.

However, there is no real ubiquitous standard for it like tcp, udp and even higher stuff like http are.

But yeah, I was planning on doing something like what you're saying, but then I'm reinventing properties of tcp that I likely won't even get right.

@halfflat I came across it from my search, but the wikipedia page didn't make it seem like it was used a lot. But maybe I was wrong... I will investigate more!

@Geoxion If it's embedded and messages, how large is one message? And how much sustained throughput do you need?

@Fast-Nop Most are pretty short, but there are some stack dumps, long logs and firmware updates as well.

Throughput is not high, like 10kBps.
I've not measured much, but the average message is less than 64 bytes.

@Geoxion At that rate and size, you could actually design a simple ACK/NAK protocol over UDP. If you use 1k packet size and have a ping of 20ms, and you use the simplest way of not sending the next packet before the ACK has arrived, that would get you to 1k/20ms, i.e. 50kB/s. Such short packets would even spare packet fragmentation on IP level.

You'd have to think a bit about how to make it robust though, with timeouts and some re-sync mechanism if the sides have gotten out of sync with each other with regard to the packet numbers. Some special "restart" message maybe.

For embedded land, can't recommend MQTT enough. Reliable pub/sub protocol that can fit on an Arduino.