I'd add in projects like Emissary, Bonfire, or https://github.com/evanp/activitypub-bot . They can all be used to build Fediverse servers from building blocks.
I think other parts might be more piecemeal. I thing HTTP Signature libraries are awesome. https://github.com/social-web-foundation/activitystrea.ms is an AS2 library that I've used a lot, even though it's weird.
https://github.com/social-web-foundation/ap-components is a Web Components library for ActivityPub objects. Also kind of cool!
I think a lot of of building a Fediverse server -- or client! -- is implementing behaviour, which an engine like Fedify can do, but most libraries aren't going to do for you.
GitHubGitHub - evanp/activitypub-bot: An ActivityPub server-side bot frameworkAn ActivityPub server-side bot framework. Contribute to evanp/activitypub-bot development by creating an account on GitHub.