Yeah. I saw the dead website earlier today and have been trying to figure out how to keep the posts working.

The fediverse software post, I could rewrite some code to gather fedidb daily (to my knowledge, they don’t have all time series data available), but their MAU math is suspect. So, not a great option, but my only other thought is basically remaking fediverse-observer and I do not have a server or potentially relevant server skill set beyond cron jobs.

For the threadiverse instances post, I can keep gathering data from the instances I already have in my database, but don’t really have another way to find new ones. Maybe, I could see if I could scrape it from fedidb (I couldn’t get their API working in the past).

In conclusion, the posts are not in a good place :(

but their MAU math is suspect

Do they do any math? I assumed it's simply additive. They get the MAU from individual instances via their crawler, then display a total?
Differences to other sites is probably due to the set of instances checked, not anything math related.

You’d think it’s straightforward and additive. I added up all their listed instances for a couple different softwares. The math was wrong and overestimated the MAU count. I checked it a few times across a couple weeks. Couldn’t quickly figure out a pattern to how the math was wrong each time.

Maybe they fixed it in the last ~6 months, but I wouldn’t count on it. I believe it’s the same Dev as pixelfed and loops. So, not sure they’ve done any work on fedidb in years (to my knowledge, they’ve focused on Loops and Pixelfed).

Edit: to clarify, adding the MAU of all their listed instances for a particular software was lower than the total MAU they reported for the software.

Edit 2: Just checked their Piefed listings and it’s still wrong. Fedidb states: ~7.0k, actual addition: 6858 MAU.

Have you contacted them about the discrepancy? Ideally FediDB could just become the new Fediverse Observer.

Missing out on accurate data about Fediverse growth would really suck. 😕

I just confirmed the problem still exists and messaged them via mastodon about it.

Probably didn’t let them know originally because I didn’t have a mastodon account at the time and already had a good option (fediverse-observer).

Differences to other sites is probably due to the set of instances checked

fedidb.com/software/lemmy?search=lemmy.world&stat…

Fedidb doesn’t seem to include lemmy.world, which seems like a substantial omission, as I believe that it’s the largest Lemmy instance.

EDIT: And my own home instance, lemmy.today, also doesn’t seem to be on there:

fedidb.com/software/lemmy?search=lemmy.today

The largest user count server that they seem to query is lemmy.ml:

fedidb.com/software/lemmy?search=lemmy.ml

EDIT2: Your home instance, kbin.earth, does seem to be on there:

fedidb.com/servers/kbin.earth

That is interesting to know. It doesn't look like lemmy.world has opted out, judging by their robots.txt.

Oh, that’s a thought. I didn’t think of checking robots.txt.

Lemmy.today does have a blanket ban on all bots in its robots.txt, which I assume was put into place to help try to reduce (sporadically severe) scraping load. I’m not sure that the admin, @mrmanager@lemmy.today, realized that it might have cut it out of Lemmy statistics gathering.

Yeah I know, but its worth it. :)

I knew to check robots.txt because of fedidb's Data Sources page, which specifies what data they get, their user agent, and how to opt-out via robots.txt.

One thing I'm noticing in regards to lemmy.world, I wonder if they ever disallowed access to the nodeinfo endpoint in their robots.txt in the past. They don't currently, but the wording suggests that having that rule at any point will forever remove the instance from fedidb. Which could explain what we're seeing for that instance.

Man that sucks. But also because I never have before, I want to thank you for all your labor making those posts! I find them very interesting.

Glad you enjoy them :)

Just need to figure out how to keep them going properly. I have a couple thoughts I’ll look into tomorrow. Hopefully one of them bears fruit.

Good luck!

Oof the stats shown in your last post there make the Threadiverse look very unhealthy.

For one, the total Lemmy MAUs falling down to the <30k range (at the high mark during Rexodus it hit ~65K, and we held steady at ~40-45K for a long time it seemed, iirc).

And for another, if I am interpreting the HYAU correctly then literally thousands of people who had accounts here are simply walking away and never coming back (then hundreds for PieFed too, though still net negative). This is crucial bc these are not people who were too daunted by the complexity of learning about the Threadiverse, picking an instance, making an account on it, and joining communities - they were already here, and now they are not here anymore.

Perhaps they are avoiding coming here (even once) during the upcoming season of Western elections, with all the contentious content that is already flooding this place leading up those, but then they might come back later? Given past trends, I doubt it, though anything is possible.

The Threadiverse looks to be slowly shrinking. Perhaps the existing people on it post more often, but with an ever-diminishing pool of recipients for that content. This will lead to a feed-forward cycle where even fewer will want to join.

And to be clear I don’t blame those people who left in the slightest - I find myself visiting here far less often than I used to as well. There’s only so much politics that I want to argue about before I find myself missing the ability to discourse on some other, any other, topic.

I think I can alleviate some of your worries. The instance post only shows the top 20 instances for a software (Lemmy has 287 active, Piefed: 59, Mbin: 16).

My Fediverse Growth Post is a better indicator for the Threadiverse’s health. From there you can see the Threadiverse is only down 2.1% from a year ago (-847 MAU). From what I’ve seen over the last >6 months, is that the Threadiverse, and most of the fediverse, grow in short spurts and then slowly declines until the next spurt.

So, unfortunately, we aren’t having continuous positive growth, but I’d say we are stable. I think stable is fine as all the fediverse developers are still adding functionality/interconnected-ness (’this isn’t our final form’).

For HYAU (Half-Year Active Users), I think it’s important to remember that lots of times people move instances, so their old account will eventually stop being counted as active (I personally have 2 old accounts). From the Fediverse post, you can see there were 42889 MAU 6-months ago and 42276 MAU 3-months ago (-613 MAU). So, even though you may see >1000 HYAU drop off in a given week, the total number of active users over that next 3 months stayed relatively the same (-1.4%, not -28% (-1000 MAU x 12 weeks)).

I think the most important information in the instance post is MAU/HYAU (%) ("Stickiness"), which indicates how well an instance holds onto its users (they keep coming back). I think >60% should be the goal of every instance (67% is currently the peak for non-small instances (>30 MAU)).

It might be worth a study to figure out why there is such a discrepancy across instances. For example, is it correlated to UI issues/customizability, reliability/up-time/longevity, admin/mod behavior, instance rules, federation/de-federation, inter-instance activity (communities/posts/comments), etc.

Regarding politics, I can’t really say what an average or new user experiences. I’ve curated/blocked massive amounts of political content. It might be worth having an instance that initially auto-blocks all politics communities for a new user. I don’t recall exactly, but I think Tesseract? was like that, but the dev/admin was sketchy about what they were doing and then melted down when called out for hiding things. So, it seems the idea had some popularity, but needs/ed a better implementation.

Thank you for your detailed response.

So the “Sum” mentioned at the bottom of each section is only for those shown within the section - the top 20 - and not expanded for all of the category? e.g. Lemmy’s MAU went down from 30817 to recently 29683, and every other metric (even for the most part the other post you mentioned for Fediverse Growth) also shows decline, but you are saying that despite that, the numerical amount is not that much, and we are mostly stable?

That is very good to hear:-).

Here is an example discussion of a community asking its members if it wants to migrate to Lemmy: www.reddit.com/r/…/should_we_switch_to_lemmy/. A few examples that seem representative to me:

I tried switching to Lemmy a long time ago, but that place is toxic and full of weird people.

There are big pockets of radical tankie communities on Lemmy that like trolling and being toxic

There are a lot more reasons than this but having spent the last 3 years on the fediverse alternatives, and just seen the shit that happens over there, I feel very confident in saying it will not ever serve as a serious Reddit alternative. It will serve as something, but not Reddit

Lemmy is shit. So is reddit but Lemmy is shittier

Here’s another example from one month ago: reddit.com/…/we_should_switch_to_something_else/

Almost nobody uses other forum apps like Lemmy. We don’t want to run two forums and scatter discussion across two places. Collectively the mods here already spend about 2 hours every day helping users on this subreddit and we don’t want more.

Or here’s a third example, from 2 months ago: www.reddit.com/r/oldreddit/…/thoughts_on_lemmy/

Opinions there run the gamut from useless due to no content to tribal politicking scaring them away from the entire federated model, though some people like it, yet don’t come here due to too few features in comparison to Reddit (with UBlock Origin tweaks to keeps some of its more insane issues in check).

We here, who have the early adopter mindset, are willing to put in the time and effort to learn how and what to block. But e.g. the most popular 3rd-party app Voyager does not - by default - show the instance name of each user account, so a new joiner won’t be able to easily distinguish when e.g. someone from the tankie triad is trolling them vs. someone attempting to converse in good conscience. Wish and hope as we might - without putting in any actual efforts to change anything mind you - we simply cannot get away from the en.wikipedia.org/wiki/Paradox_of_tolerance principle.

We are a tankie / troll bar, and the vast majority of mainstream non-technical normal folks tell other people that they very much do not enjoy visiting here as a result. Admiral Patrick very much had a bad implementation to a somewhat good idea: being forced to view toxic content should be an option that users have some control over, somehow, even if that choice is disabled by default. e.g., it would be nice if at least one singular major instance would defederate from lemmy.ml, or in the case of a PieFed instance apply an instance block by default - but sadly, literally none of them do. Of course, they would simply use their alts to get around any such block, but otherwise we are back to the Paradox of Intolerance again: with next to zero distinction made between trolls vs. non-trolls, why would most people want to come here to discuss things with us in our troll bar?

No problem. Happy to provide insight into my workings :)

So the “Sum” mentioned at the bottom of each section is only for those shown within the section - the top 20 - and not expanded for all of the category? e.g. Lemmy’s MAU went down from 30817 to recently 29683, and every other metric (even for the most part the other post you mentioned for Fediverse Growth) also shows decline, but you are saying that despite that, the numerical amount is not that much, and we are mostly stable?

Yes and yes.

Yeah, the snippets and links you provided match my understanding of the situation. I don’t recall who or where, but I recall someone saying they were having a much easier time recruiting people to Piefed rather than Lemmy, due to Lemmy already having an established reputation. That being said, I expect the Lemmy 1.0 launch will cause a significant bump.

… don’t come here due to too few features in comparison to Reddit …

I found the user saying that, but they didn’t elaborate on what missing features they wanted. It’s been years since I used Reddit, do you have any clue what features they may be wanting?

… Voyager does not - by default - show the instance name of each user account …

I recall running into this issue and being confused, before I found the setting. Do you know if this is still the default? If so, and the Dev hasn’t given a specific reason for the decision (I can’t immediately find a post about it), I don’t mind making a post to ask them to switch the default.

… it would be nice if at least one singular major instance would defederate from lemmy.ml, or in the case of a PieFed instance apply an instance block by default …

Agreed.

… they would simply use their alts to get around any such block …

Probably, but I think it would be an order of magnitude fewer users encountered, than if the block didn’t exist. If they did use alts to get around the block, their alt’s instance’s admin could be contacted. My understanding is that the “troll behavior” on .ml is allowed, may be even encouraged, by their admins/devs. So, different admin may not have a tolerant attitude toward it. Along with this, as more communities spread out from the flagship and most popular instances (.ml, .world, .social), I think things will get less toxic too.

As an aside, over the last 3-months, it looks like most of the “politically” focused instances are the ones on the heavier declining side of the range. The instances declining the least or even growing are on the “unaligned” side of things.

do you have any clue what features they may be wanting?

Some people for some reason - particularly younger ones - seem to demand usage of a 3rd-party app. So my guess is that they may have meant things like user and post flairs, polls, emoji reactions, categories of communities (multi-reddits), etc., which PieFed technically offers but most 3rd-party apps have yet to catch up to utilize those capabilities. For example Voyager seems by far the most commonly used one, but it only offers nominal beta-level support for PieFed.

And I don’t know how PyLova may change the situation still further - it offers even more features than PieFed, though the latter keeps incorporating most of those changes back into its main codebase, so at the moment they are still fairly similar.

Do you know if this is still the default?

Yes, unfortunately. I was trying out several 3rd-party apps like Summit and Voyager to see what all the fuss is about, and unfortunately some of those aspects that I would consider crucial to my own usability - like that one - are buried somewhere in the settings, disabled by default. I can’t tell you how many times I began to craft a response to someone, only to notice that they are on Lemmy.ml at which point I abandoned my endeavor as it seemed futile given that additional information. Echo chambers do exceedingly bad things to the people trapped inside of them.

I also thought about making such a post, but I can see from the devs’ perspective that turning on the display of instance names in Voyager is not a feature that is universally good. For one thing it doesn’t always work, especially in replies deeper into a chain, and especially dependent upon the length of the actual username itself. And for another, does it really matter? For someone who knows the culture here it can greatly aid, and arguably it helps someone learn about the culture here, but the largest majority of the time the only thing that matters is “belongs to tankie triad” vs. not, after which distinctions such as “is on lemmy.world” vs. let’s say sh.itjust.works are much finer-grained nuances that rarely rise to being necessary. In other words, if there was some major instance (like PieFed.zip) that would place a user-level block onto Lemmy.ml just like it does to hexbear.net and lemmygrad.ml, then display of instance names following the username is no longer necessary, for new joiners to the Threadiverse. Hence, from their perspective, I could see a resistance to that solution being used to deal with the underlying problem imperfectly.

About instance blocking, I recently discovered that programming.dev has a number of community-level blocks put into place to help a new user experience a more drama-free version of the Threadiverse. You can see their full listing at legal.programming.dev/docs/hidden-communities/, although I note that while it includes explicitly “political” ones, it misses ones that end up having a huge bias towards inclusion of political content regardless of the statement in their sidebar - like !memes@lemmy.ml - which can be quite a turn-off to people, especially those in the Western World, and those undecided whether to join or not, when such content appears and they don’t know yet how to distinguish it from e.g. identically-named communities on different instances (which are also extremely heavily political in those other locations as well, though with less content advocating for the literal murder of every person currently living inside of a Western civilization).

Probably, but I think it would be an order of magnitude fewer users encountered, than if the block didn’t exist.

Agreed - solutions need not be perfect to have at least some effect, I guess I meant here in comparison to like a “reputation” indicator such as PieFed offers, which works regardless of instance affiliation, as in someone who receives 10x more downvotes than upvotes.

As an aside, over the last 3-months, it looks like most of the “politically” focused instances are the ones on the heavier declining side of the range. The instances declining the least or even growing are on the “unaligned” side of things.

Oh wow, that’s extremely interesting. Though all smaller numbers of course. As the technology continues to improve, the overall experience can get better, it is just frustrating to me that 3+ years later things look mostly the same as the original discussions people have where they would come to the Threadiverse using Kbin but not Lemmy, and now it seems like for the most part they simply will not come period. So it’s interesting to see that even countering with only a trickle, that is not universally true:-).