Contrary to what some people say, #HolosDiscover is not a scraper. It speaks #ActivityPub, and every deleted or edited post is immediately updated in the database.
It relies on the #Fediverse indexable flag, with stricter rules: accounts with #NoBot or #NoIndex in their profile, and locked accounts, are excluded. Only public posts are indexed.
I recently fixed a bug where the #NoIndex tag in bios was not respected. Sorry about that.
— Open image attachment 1 of 1HolosDiscover overview stats. 20.9M posts indexed. 6.2M posts deleted (about 30% of indexed posts) and 712K posts updated (about 3.4%), all reflected in the database. 90K users followed, 3.7K opted out, 8.6K instances known.
