Whether you’re really passionate about RPC, MQTT, Matrix or wayland, tell us more about the protocols or open standards you have strong opinions on!
RSS. It’s still around but slowly dying out. I feel like it only gets added to new websites because the programmers like it.
90% of the bullshit mass emails at my work could be an RSS feed.
“THIS WHOLE MEETING COULD HAVE BEEN AN RSS FEED!”
Theres quite a few sites that still use it and existing ones in the Fediverse have it built in (which is really cool). But your right, the general public have no concept of having something download and queue up on a service rather than just going to the site. And the RSS clients are all over the place with quality…
WebSub (formerly PubSubHubbub). Should have been a proper replacement for RSS with push support instead of polling. Too bad the docs were awful and adopting it as an end user was so difficult that it never caught on.
I wouldn’t say that it never caught on. I run a feed reader and ~6% of feeds have WebSub. Most of these are probably wordpress.com blogs which include it by default.
YouTube also sort of supports it, but they don’t really follow the standard so I don’t think it counts.
But the nice thing about WebSub is that it is sort of an invisible upgrade to the existing feed (or any other HTTP URI) so it just works when blogs enable it.
Most major feed reader services support it. One problem is that you need a stable URL to receive the notifications. So it is hard to make work with client-side readers. But I don’t think there is really a way around this other than holding a connection open to every feed you follow. So I would say that it does its job well. I don’t really see a need to get to 100% adoption or whatever. If you have a simple static-site blog that updates every month or so I don’t think it is a big deal that it doesn’t support WebSub.
I still want something push based (without paying for those rss as a service)
deleted by creator
Oh neat! I didn’t know this existed. By any chance, do you know of any RSS readers that have implemented it?
Literally nothing uses rssCloud. WebSub is what you want.
IPv6
I mean, why the hell is IPv4 still a thing??
Because ipv6 is yucky
Try to remember a handful of them
Removed by mod
On the Internet, no. On my home LAN? Absolutely. I disabled all IPv6 at home.
Removed by mod
I hear you on this! Took me a whole day to get my router to delegate IPv6 properly. I’m sure that had it been better adopted, I wouldn’t be having such a hard time.
Unified Push.
Unbelievable that we have to rely on Google and co for sth as essential as push messages! Even among the open source community, the adoption is surprisingly limited.
Nobody knows about unifiedpush. Last time I checked, their Linux dbus distributor also wasn’t ready. There has to be a unified push to get it adopted.
Removed by mod
Fuck Unified Push. Just use the Web Push standard. https://www.rfc-editor.org/rfc/rfc8030
It is what is used for browser push messages, is already widely supported. Is compatible with existing push infrastructure and users and is end-to-end encrypted. IDK why Unified Push felt the need to create a new protocol when a perfectly good one already existed.
Although there is no “client side” spec. The Unified Push client side could be useful. But they should throw away their custom backend protocol and just use Web Push.
Markdown. Its only in tech-spaces that its preferred, but it should be used everywhere. You can even write full books and academic papers in markdown (maybe with only a few extensions like latex / mathjax).
Instead, in a lot of fields, people are passing around variants of microsoft word documents with weird formatting and no standardization around headings, quotes, and comments.
I think Obsidian and Logseq are helping to change this.
Man, I’ve written three novels plus assorted shorter form stories in markdown.
There’s a learning curve, but once you get going, it’s so fluid. The problem is that when it comes time to format for release, you have to convert to something else, and not every word processor can handle markdown. It’s extra work, but worth it, imo.
Just set up pandoc and Bob’s your uncle. It’ll convert markdown to anything. You’ll never have to open another word processor.
Nice! Thanks for the tip!
Edit: holy shit, how have I never run across that before? That’s a brilliant program right there.
Pandoc + [your markdown editor of choice] is magic. Some editors even come with Pandoc as a dependency so you can export to more or less anything from the GUI. I think GhostWriter and Zettlr at least (I honestly can’t be sure, I’ve changed editors so often and now I just have some Pandoc conversion scripts in my file manager menu).
Markdown is terrible as a standard because every parser works differently and when you try to standardize it (CommonMark, etc.), you find out that there are a bajillion edge cases, leading to an extremely bloated specification.
Most ppl have settled on Commonmark luckily, including us.
Commonmark leaves some stuff like tables unspecified. That creates the need for another layer like GFM or mistletoe. Standardization is not a strong point for markdown.
Have you read the CommonMark specification? It’s very complex for a language that’s supposed to be lightweight.
What’s the alternative? We either have everything specified well, or we’ll have a million slightly incompatible implementations. I’ll take the big specification. At least it’s not HTML5.
An alternative would be a language with a simpler syntax. Something like XML, but less verbose.
And then we’ll be back to a hundred slightly incompatible versions. You need detailed specifications to avoid that. Why not stick to markdown?
IOT devices shouldn’t connect to wifi. ZWave or zigbee is much better suited to IOT stuff, but it seems to mostly get adopted in very limited, locked down proprietary shit like Hue Lights.
matrix, or at least interop standards for online communications. It’s such bullshit that you make a shitty chat app, and just because it’s free and relatively featured, become the single existing monopoly of chat applications.
Like idgaf as long as i can host a bridge between discord and matrix or some shit, and you technically can, but it’s a right pain in the ass to do so.
Yup. Way too many people using different chat apps. I’ve bridged most of them but still annoying.
For business email is thankfully still pretty common. But some of them try to push you to one of the Facebook messengers.
I want an open widely used chat app ASAP.
and even with email, that’s still open. So not a huge concern, or at the very least standardized enough to make it easily interop.
But yeah, i would greatly appreciate anything that isn’t fucking discord.
IPFS I’m really glad things like nerdctl and guix support it, but I wish more things too advantage of the p2p filesystem.
Petals.Dev and hivemind ml P2P AI inference and training seem like the only true viable options to make foundational models that are owned soley by authoritarian government s and megacorps.
Matrix for federated general real time communication. (Not justs chat, video, images, but just data, with third room being on the cooler demos for what is possible)
Activity Pub for asynchronous communication between servers. The socialmedia aspect is obviously the focus and the most mature, but I’m also excited for things like Forgejo (Codeberg.org) and Gitlab’s support.
I am also excited for QUIC for increased privacy of metadata and reduction of network trips.
The problem with IPFS is that kubo sucks. I used it for a while and it is always burning CPU doing nothing, failing to publish anything to the DHT and fetching files is slow. GC collects files that are supposed to be “pinned” by existing in MFS and so many other bugs all of the time.
I would love to see a new server take off that was built with quality in mind.
I think the core protocol is pretty good (although I think they should have baked in encryption) but even the mid-level protocols like UnixFS and DAG whatever are mired in constant change for no real reason as the developers jump on new fads.
Slow and requires additional tooling to run normally. Just not a lot of development on the core pieces tbh. Wasm support for example could make deployments way simpler (implement an ipfs proxy in any browser or server that supports wasm) but the ticket for that kind of died off. There is a typescript implementation, helia, that I haven’t checked out yet.
We are honestly kind of in a decentralization winter again, with ActivityPub being one of the few survivors gaining traction from what it seems. OpenSource luckily doesn’t just up and die like that, so I still have hope for some next spring.
odf/odt/ods
.md
SimpleX
Matrix
OpenPGP
Last, certainly not least… ActivityPub
- IPv6, needed for modern Internet not to collapse, would make many other important things easier. Easier to become an ISP, to selfhost, to build P2P networks, etc.
- GNU Taler, a payment protocol just look at it go: https://101010.pl/@didek/111934952208145427, or just imagine building a payment terminal of a Raspberry Pi
- Matrix, to unify chat, conference and calling apps
- some self-arranging darknet protocol becoming a norm like I2P, GNUNet or Yggdrasil, so we could have a backup when mass Internet blockage happen
I really hope matrix gets native VoIP. I saw like 2 years ago it was in beta, haven’t kept up with it though. I’d also really like voice channels like discord so my friends and I can replace discord but it seems like matrix isn’t interested in being a discord replacement
Matrix can be configured to have VoIP. I have it set up on my server. Haven’t tried it in group voice chat setting yet though. Only 1 on 1
I wish Microsoft Office would use the .odf standard by default. Or, failing that, it’d implement its own published .docx specification correctly, so other office suites can be compatible.
That’d be nice of course. Personally, I just wish everything Microsoft would wither and go away.
The entire purpose of Microsoft standardizing OOXML and implementing it wrongly in Office was to make other office suites irrelevant. ODF was already standardized and countries would have adopted it if MS didn’t do the same with OOXML. They stuffed the ISO with members supporting them to do it.
And now that OOXML is a viable standard, they implement it wrongly so that other office suites can’t be compatible with MS Office without a lot of extra effort. Any incompatibilities with MS Office will be considered as the fault of other office suites by the general public and government officials.
Expecting MS to do what’s right for the customers is putting too much faith in their nonexistent sense of ethics.
honestly: activity pub, matrix, xmpp, markdown and soo many more probably. All of these would be able to solve our walled gardens problem, but the apps with a basically monopoly don’t have much of an incentivw to implement them
FTP
Seriously guys, let’s share files the old fashioned way. Without bullshit.
I’d like to interject for a moment. What you’re referring to as FTP is, in fact, smelly hot garbage.
For context, I wrote this while waiting for a migraine to pass. I was angry at my brain for ruining my morning, and I like to shit on FTP. It’s fun to be hyperbolic. I don’t intend for this to be an attack on you, I was just bored and decided to write this ridiculous rant to pass the time.
I must once again rant about FTP. I’ve no idea if you’re serious about liking it or you’re just taking the piss, but seeing those three letters surrounded by whitespace reminds me of all the bad things in the world.
FTP is, as I’ve said, smelly hot garbage, and the infrastructure built to support FTP is even worse. Why? Well, one reason is that FTP has the most idiotic networking model conceivable. To see how crazy it is, let’s compare to a more sane protocol, like HTTP (for simplicity’s sake, I’ll do HTTP/1.1). First, you get the underlying transport protocol stuff and probably SSL. The HTTP client opens a connection from some local ephemeral port to the destination server on port 80/443/whatever and does all the normal protocol things (so syn->synack->ack and Client Hello -> Server Hello+server cert -> client kex+change cipher -> change cipher -> encrypted data). FTP does TCP too! Same same so far (minus SSL, unless you’re using FTPS). Next, the HTTP client goes like this:
GET /index.html HTTP/1.1 Host: www.whatever.the.fuck # a bunch of other headers
and you know what fucking happens here? The fucking server responds with the data and a response code on the same goddamn TCP connection. You get a big, glorious response over the nice connection you established:
200 OK # a bunch of headers and shit HERE'S YOUR DAMN DATA NERD
So that’s nice, and the client you’re using to read this used that flow (or an evolution of that flow if you’re using HTTP/2 or HTTP/3). So what does FTP do? It does one of two really stupid things depending on whether you’re using active or passive mode. Active mode is the default for the protocol (although not the default for most clients), so let’s analyze that! First, your FTP client initiates a TCP connection to your server on port 21 (by default), and then the server just sends this:
<--- 220 Rebex FTP Server ready.
ok, that kinda came out of nowhere. You’re probably using a modern client that saves you from all of the godawful footguns, so it then asks the server what it supports:
---> FEAT <--- 211-Supported extensions: <--- AUTH TLS;SSL; <--- CDUP <--- CLNT # A whole bunch of other 4 letter acronyms. If I was writing an FTP server, I'd make it swear at the user since there are a lot of fun 4 letter words
There’s some other bullshit we don’t care about right now, although highlights include sending the username and password in plain text. There’s also ASCII vs binary mode. WE’LL GET BACK TO THAT. :|
So then we want to do a LIST. You know what happens in active mode? Your computer opens up some random fucking TCP port. It then instructs the FTP server to CONNECT TO YOUR GODDAMN COMPUTER. Your computer is the server, and the other side is now the client. I would post a more detailed overview of the FTP commands, but most servers on the internet disable active mode because it’s a goddamn liability. All of the sudden, your computer has to be internet facing with open firewall ports, and that’s just a whole heap of shit.
I’m probably not blowing many minds right now because people know about this shit. I just want to mention that this is how FTP was built. The data plane and control plane are separate, and back in 19XX when this shit was invented, you could trust your fellows on ARPANET and NAT didn’t exist and sure HAM radio operators here’s the entire goddamn
44.0.0.0/8
block for you to do packet switched radio. A simple protocol for simple times, back before we knew what was good and what was bad.So, active mode sucks! PASV is the future, and is the default on basically all modern clients and servers! Passive mode works exactly the same as the above, except when the client goes to
LIST
, the server opens some random TCP port (I’ve often seen something like 44000-44010) and tells the client, “hey you, connect to 1.2.3.4:44000 to get you your tasty data.” Sounds great, right? Well, there’s a problem that I actually touched on in my last paragraph. Back when this dogshit was first squeezed out in the 70s, everyone had a public address. There were SO MANY addresses! 4 billion addresses? We’ll never use all of those! That is clearly not the case anymore. We don’t have enough addresses, and now we have this wonderful thing called NAT.Continued in part 2.
PART 2.
NAT, much like the city of Phoenix, is a monument to man’s arrogance. Fuck NAT and fuck FTP. If your FTP server is listening directly on a public IP address hooked up directly to a proper router, then none of this applies. If you’re anything like me, the last company I worked for (a small startup), or my current company (many many thousands of employees making software you know and may or may not hate, making many billions of dollars a year), then the majority of your servers are living in RFC1918 space. Traffic from the internet is making it to them via NAT (or NAT with extra steps, i.e. L4 load balancers).
A request comes in for $PUBLIC_IP TCP port 21 and is forwarded to your failure of a boxen at 10.0.54.187. Your FTP server is a big stupid idiot and doesn’t know this. It thinks that it’s king shit and has its own public IP address. Therefore, when it’s deciding what ADDR:PORT it’s going to tell the stupid FTP client to connect to, it just looks at one of the adapters on the box and says “oh, I’ll tell this client on the internet to connect to 10.0.54.187:44007” and then I fucking cry. The FTP client is an idiot, but the IP stack on the client’s home/business router is not and says “oh, that’s an address living in RFC1918 space, I shouldn’t send that out over the internet” and they don’t get the results of their LIST.
So, how do you fix this? Well, you fix it by not using FTP. Use SFTP USE SFTP USE SFTP FOR GOD’S SAKE. But since this world is a shit fucking place, you have two options. The best option is to configure your FTP server to lie about its IP address. Rather than being honest about what a fool it is, you can tell it to send your public IP address to the client rather than the network adapter IP address. Does your public IP address change? Fuck you, you get to write a daemon that checks for that shit, rewrites your FTP server config, and HUPs the bastard (or SIGTERMs it if your server sucks and can’t do a live config reload).
Let’s say that you don’t want to do that. Let’s say you work at a small company with a small business internet plan that gives you static IPs but a shitty modem. Let’s say that you don’t know what FTP is or how it works and your boss told you to get it set up ASAP and it’s not working (because the client over in Bendoverville Arkansas is being told to connect to a 10.x.x.x address) and it surely must be your ISP’s fault. So you call up Comcast Business/AT&T/Verizon/Whoeverthefuck and you complain at their technicians for hours and hours, and eventually you get connected to a human that knows what the problem is and tells you how to configure your stupid FTP server to lie like a little sinner. The big telco megacorps don’t like that. They don’t want to waste all those hours, and they don’t want to hire too many people who can figure that shit out because it’s expensive. You wanna know what those fucking asshole companies did?
Continued in part 3.
PART 3.
They made their STUPID MODEMS FUCK WITH THE FTP PACKETS. I have personally experienced this with Comcast Business. The stupid piece of shit DOCSIS modem they provide intercepts the FTP packet from your server saying “oh, connect to this address: x.x.x.x:44010” and they rewrite the fucking address to the public IP. There is no way to turn just this horse piss off. Now, for average business customers, this probably saved Comcast a bunch of money in support calls. However, if you’re using the so-called bridge mode on that degenerate piece of shit-wrapped-silicon (where rather than allowing the modem to give you a DHCP address, you just configure your system to have one of the addresses in the /29 space and the modem detects that and says oh okay don’t NAT traffic when it’s going to this address, just rewrite the MAC and shunt it over the right interface), then something funny happens. The modem still rewrites the contents of the packet, but it uses the wrong fucking IP address! Because the public IP that your server is running on is no longer available to the modem, the modem just chooses another fucking address. Then, the client tries to connect to 1.2.3.5 instead of 1.2.3.4 where your server is listening, the modem says “hey I’m 1.2.3.5 and you can fuck off, I’m dropping your SYN for port 44010”, and I get an angry call from the client asking why they can’t download their files using this worthless protocol. I remember having a conversation like this:Me: “Just use SFTP on port 22!”
Client: “No! FTP is faster/more secure/good enough for my grandfather good enough for me/corporate won’t allow port 22.”
Me: “Comcast is fucking me right now. What if we lied and served SFTP over port 21?”
# we try it
Client: “It’s not working! I can’t even connect!”I couldn’t connect either. I couldn’t connect to anything. Trying to do SFTP over port 21 caused the stupid fucking modem to CRASH.
Are you starting to see what the problem is? It’s like Microsoft preserving bugs in Windows APIs so that shitty software doesn’t break, and then they end up doing crazy gymnastics to accomodate old shit like the Windows 8 -> Windows 10 thing where they couldn’t use “Windows 9” because that would confuse software into thinking it was running “Windows 95” or “Windows 98”. FTP has some bugfuck crazy design decisions that we’ve collectively decided to just “work around,” and it leads to fucking gymnastics.
Speaking of bugfuck crazy design decisions, FTP’s default file transfer mode intentionally mangles data!
Continued in part 4.
PART 4.
You expect a file transfer program to reliably and faithfully transfer your files, byte-for-byte, from one system to another. FTP spits in your face and shits on your chest. You know how Linux uses LF (i.e.
\n
) for newlines and Windows uses CRLF (i.e.\r\n
) for newlines? Pretty annoying, right? Well, FTP’s ASCII mode will automatically rip off those\r
characters for you! Sounds pretty sweet, right? Fuck no it’s not. All of the sudden, your file checksums have changed. If you pass the same file back to a Windows user with a different and more sane file transfer system, then they get a broken file because FTP didn’t mind its own fucking business. If you have a CRLF file and need an LF file, just explicitly usedos2unix
. Wanna go the other way?unix2dos
. The tool has been around since 1989 and it’s great.Now, what if you’re not transferring text, but instead are transferring a picture of a cute cat? What if your binary data happens to have 0x0D0x0A somewhere in it? Well, ASCII mode will happily translate that to 0x0A and fucking ruin your adorable cat picture that you were going to share with your depressed significant other in an attempt to cheer them up. Now the ruined JPEG will remind them of the futility of their situation and they’ll slide even deeper into cold emptiness. Thanks, FTP.
You can tell your client to use binary mode and this problem goes away! In fact, modern clients do this automatically so your SO gets to see the adorable fuzzy cat picture. But let’s just stop and think about this. Why use a protocol that is dangerous by default? Why use a protocol that supports no form of security (unless you’re using fucking godawful FTPS or FTP over SSH)? Why use a protocol that is so broken by design that small business hardware has been designed to try to unfuck it? Is it faster? I mean, not really. SFTP has encryption/decryption overhead, but your CPU is so fast that you’d need to transfer at 25+ Gb/s to notice it. Is it easier? Fuck no it’s not easier, look at all of the stupid footguns I’ve just mentioned. Is it simpler? The line protocol is simple, but so is HTTP, and HTTP has a much simpler control flow path (merging the data and control planes is objectively the right thing to do in this context). And shit, you want a simple protocol for cases where you don’t have a lot of CPU power? Use fucking TFTP. It’s dogshit, but it was intentionally designed to be dogshit so that a fucking potato could receive data with it.
There is no task that is currently being done with FTP that couldn’t be done more easily, more securely, and more quickly with some other protocol (like fucking SSH and SFTP, which is now built into fucking Windows for god’s sake). Fuck FTP.
Have you considered publishing that as a book? (/s)
You are insane… in a good way. I love it. Fantastic read and I had to chuckle a few times.
I’m glad that my grumpy migraine ramblings brought someone some joy!
I read the first two and kinda gave up my dude. Here’s my deal. I get that it’s bad under the hood. What else can I use that lets me and a friend pretend we just have folders in each other’s computers with just a port forward, IP, and a password?
You can set up SFTP with a password.
Or WebDAV isn’t that awful.
That’s not even the type of setup you should use. Use a VPN of the type designed for games and IoT stuff, like ZeroTier, n2n, and more. Then you set up a local file share using something like Samba, only accessible by the people who can connect to your local network via the VPN.
The public facing VPN code will be MUCH more hardened against attack than your typical sharing tool with port forwarding.
I’m less worried about attack than barriers to sharing.
In that case, I’d like to chime in and add NFS to this list. The often overlooked jewel of the glorious past days. /j
OpenTelemetry everywhere please
peer to peer, i would be happier thitking that every time i open somo application, i’m helping it, like i2p
Ever heard of IPFS? I really hope that will take off some time.
Unfortunately the reality of IPFS is that despite its huge funding it was poorly designed from the start and still to this day has much slower loading times then my I2pd instance (despite i2p transmiting messages through multiple encrypted proxies), to the point where the company working on the rust implementation determined it was so bad they had to scrap the whole thing to make something that actually worked. Not to mention that I managed to have my server taken over by some kind of malware by downloading a particular piece of content.
Thanks, that was an interesting read! I always felt IPFS wasn’t ready yet, but the value it tries to provide of being a file system, I’ve found no real alternative to. Very good to read that iroh is willing to look beyond the IPFS spec to provide its values with better performance. I hope it works out.
i wish all the big players would agree on one of the many open chat and IM protocols. it’s like kindergarten where the toddlers don’t want to share toys
Was it really back in 2009 that both Google and Facebook used XMPP compatible chats? Those were the days.
Facebook had an XMPP client API. It didn’t federate (and wasn’t really true XMPP resulting in many quirks).