Of all the ActivityPub servers that have ever been publicly available, how many do you think are still up today?
0 to 25%47%
26% to 50%35%
51% to 75%11%
76% to 100%5%
Replying to @evan@cosocial.ca
Hey, all. Very interesting. So, I've mentioned this a few times in the comments, but I've done a survey of about 85,000 servers that have been seen by existing live servers. Of those servers, about 32% reply successfully to a Nodeinfo request. Nodeinfo is a per-server statistics endpoint.
Replying to @evan@cosocial.ca
Not all ActivityPub servers support Nodeinfo. Of that same data set, about 8% give a successful 404 or 410 response for Nodeinfo. Some percentage of those are going to be non-nodeinfo Fediverse servers, and with a bit more work I could probably figure out which are which.
Replying to @evan@cosocial.ca
So, by my estimate, of these servers, about 32-40% of Fediverse servers seen by other live servers are alive today. The rest return errors, don't resolve, have bad SSL certs, and so on.
Replying to @evan@cosocial.ca
Now, there's some dark matter out there: servers that never connected to our large connected component on the Fediverse. We can't know if they're up or down or what. They're unreachable.
Replying to @evan@cosocial.ca
Regardless, I'm pretty confident in using the 32-40% number. So, that's in the 26-50% quartile, and that would be my answer.
Replying to @evan@cosocial.ca
My guess is that @dansup might have some better data than I do.
Replying to @evan@cosocial.ca
More data about servers:
- 27K live servers.
- 47% have one user.
- 55% of all users are on the top 20 servers.
- 88% of all users are on the top 100 servers.
- 99% of all users are on the top 1000 servers.
Replying to @evan@cosocial.ca
I think there might be ways to find the dark unconnected Fediverse servers using something like Common Crawl.
Replying to @evan@cosocial.ca
Comment
@evan There must be an enormous number of one-person servers, so the number of surviving servers need not tell much about the Fediverse as a whole.
Replying to @mrdk@mathstodon.xyz
Comment
@mrdk As far as I can tell, about 47% of live servers are single-user.
What do you mean that the lifetime of servers doesn't tell much about the Fediverse as a whole?
Replying to @evan@cosocial.ca
Comment
@mrdk Also, as far as I can tell, about 55% of all Fediverse users are on the top 20 biggest servers. About 88% are on the top 100; about 99% are on the top 1000 biggest servers.
Replying to @omkar_foss@mastodon.social
Comment
The method is adding the user count for the top N servers and then dividing that by the total user count for the Fediverse (or at least all known servers).
You can get the user count from a server using Nodeinfo.
Replying to @evan@cosocial.ca
Comment
@evan Thank you!
Replying to @omkar_foss@mastodon.social
Comment
@omkar_foss not all Fediverse software supports Nodeinfo, so it's an estimate rather than a measurement.
You also have to know that a server exists. There's a Mastodon API endpoint that shows which other servers a server has seen; you can use that to do a breadth-first survey of the network.
Replying to @evan@cosocial.ca
Comment
@evan sure, I think an estimate might be good enough to understand Fediverse health.
I guess /instance/peers might be the Mastodon API endpoint that you mention: https://docs.joinmastodon.org/methods/instance/#peers
docs.joinmastodon.org/methods/instance/#peersReplying to @omkar_foss@mastodon.social
Comment
@omkar_foss Yes, I think that's it!
Replying to @mrdk@mathstodon.xyz
Comment
Yes, perhaps might've been better to make the distinction, and ask for multi-person servers specifically.
The term "server" itself is imho an unfortunate case of a lower-level abstraction (that of the installed base) creeping into a high-level one (that of the unified social network stratum). It is a passed station, but a multi-account server/instance might simply be called a Hub.
"How do you join the fediverse, Self-hosted or via a Hub?"
"My actors are all on Hubs."
(Actor is also kind of a technical name, but Person doesn't match, and Identity is awkward. Account is possible, still technical. Fedizen might be used, but rings a bit weird.)
Replying to @smallcircles@social.coop
Comment
@smallcircles @mrdk why?
Replying to @evan@cosocial.ca
Comment
@smallcircles @mrdk Also, I ask the questions I want answers to. I don't like being told by other people what questions I should ask. If *you* want to ask some different question, you should definitely do it! Let me know, and I'll boost it.
Replying to @smallcircles@social.coop
Comment
@smallcircles @mrdk I wonder what the implications you read into the poll that made you think that single-user servers shouldn't count. What do you think it is "really" asking, or should be really asking?
Replying to @evan@cosocial.ca
Comment
@evan OMG I have this problem SO OFTEN trying to get measurement papers published in academic venues, we answer some interesting question and the reviews are basically "but why didn't you answer this OTHER question", and I just want to respond to say *this is an interesting question and I have answered it, if you want to answer this other question, how about you answer it*
Replying to @ricci@discuss.systems
Comment
@ricci i asked a few times and no one seemed to want to say why asking about multi-user servers is more important than asking about all servers. I think, but don't know, that they want to have some answers about the viability of the volunteer-run community server model? I'm not sure.
@julian Polls look great in NodeBB. Great work!
Replying to @evan@cosocial.ca
@evan Do you have the actual answer to this one?
Replying to @anca@mastodon.xyz
@anca I have *an* answer. It's not possible to determine every single server ever stood up, but I've got records for about 85,000 servers that have communicated with currently live servers. So, it's an OK estimate.
Replying to @evan@cosocial.ca
@anca servers that only ever communicated with other servers that have since gone offline wouldn't show up in my data.
Replying to @liaizon@wake.st
@liaizon @evan well... what I'm doing is using a list of servers that are currently up, and just getting nodeinfo for them, so I do have a list of *some* historical servers that are gone now, but I doubt it's anywhere near as good as Evan's data.
Someone who might have good data is @minoru who has been crawling the lists of federate servers for several years now to make https://nodes.fediverse.party/ (the list I use), and would have probably have a bunch of no-longer-reachable nodes in his database, including ones that were only known by nodes that are themselves no longer up, etc.
nodes.fediverse.party/
Replying to @ricci@discuss.systems
@ricci So, my crawler has been running for 5 years, and it also pulls data from other sources like the-federation.info, fediverse.observer, fedidb.com etc. Its database has 44142 alive nodes and 777384 dead ones, so the answer is 5.4%. If .gab.best and .cheeseland.xyz are excluded, then only 98621 nodes are dead and the answer is 30.9%.
Someone in the thread suggested that nodes might be pruning their peers lists from dead nodes. Judging by the negligible difference between Evan's number and my number, no pruning is done.
BTW, my crawler currently tracks 22 newly discovered nodes, i.e. ones that were seen in someone's peers list but whose NodeInfo hasn't been checked yet. I'm happy the Fediverse keeps growing _^
Replying to @evan@cosocial.ca
@evan i would love a breakdown on the one user servers if you did a followup to this. are these actual servers in actual use, or developer test servers?
Replying to @kim@post.lurk.org
@kim I think I can do that!
Replying to @evan@cosocial.ca
That's interesting. I figured there'd be a LONG tail of servers that someone spun up, ran for a few days, and then shut down.
But maybe there are, but live servers wouldn't remember them?
It seems like it would be a good design for them to eventually forget inactive accounts?
I know that some accounts are listed by my server as "inactive", simply because no one on my server follows them anymore (but following back to the origin server, they are still there and active).
Replying to @TerryHancock@realsocial.life
@TerryHancock 55k dead servers seems like a pretty long tail to me.
Replying to @evan@cosocial.ca
@evan do you know what the actual numbers are?
Replying to @toddsundsted@epiktistes.com
@toddsundsted I have some ideas but there are some unknown unknowns. I feel like 35-40% is a defensible number.