How we score and measure

Every tool in the index is described with the same fields, in the same order, from the same sources. That is the whole trick. A comparison is only meaningful if both sides were measured the same way, and most software comparisons are not, because each entry was written from whatever its own marketing page happened to emphasise.

Where the facts come from#

Factual fields are checked against primary sources, in this order of preference:

1. The project's own documentation, including its requirements, installation and backup pages.
2. The project's repository: the LICENSE file, the release notes, the changelog, the sample configuration files, the published container manifests.
3. The project's own announcements.

Aggregator sites, listicles and other people's reviews are not sources. Where a number comes from the project itself, the profile says so. Where a number is ours, the profile says that too. Every profile ends with the source list used to build it.

Two rules follow from that, and they are the ones that cost the most work:

• If we cannot source it, we do not print it. A missing field is better than a confident guess. Several profiles say "not recorded" for a first-release year because the project's own history does not settle it.
• We re-check what we think we know. A large share of what is written about this niche is two or three years stale, including licence claims, maintainership claims and version numbers. Several entries in this index contradict the current top search results because the current top search results have not been updated.

The operational load score#

A whole number from 1 to 5. It answers one question: how much ongoing attention does this cost a competent single operator, after it is running?

Deliberately, this is not a difficulty-of-installation score. Nearly everything here installs in an evening. The score is about the following two years.

It is an editorial judgement, not a measurement, and we say so wherever it appears. What makes it useful is that it is applied consistently and that the reasoning is printed on every profile in the ops_load_reason field, so you can disagree with a specific score on the specific grounds we gave.

Memory figures#

Two numbers per tool, both in megabytes:

• Idle: freshly started, small library, nobody using it.
• In ordinary use: normal single-household activity.

These are working figures for planning, not benchmarks. Where a project publishes its own documented minimum, we use that instead and the profile note says which it is. For a hypervisor or a NAS operating system, the figure is the documented minimum system memory, because "idle" is not a meaningful concept for a base layer.

The sizing tools deliberately use the ordinary-use figure and not the idle one. A machine sized to idle swaps the first time somebody actually opens the application.

How backup shape is derived#

This is the one field on the site that is computed rather than researched, and the derivation is fully mechanical:

Many projects support several engines, and their own documentation almost always names the default first: "SQLite by default, PostgreSQL supported". The derivation therefore matches on the engine that appears earliest in the declared datastore string rather than on a fixed order of precedence, so a tool you are running on its default SQLite is filed as SQLite rather than as Postgres. Where a store that looks file-shaped also names a real database, the database wins, because that is the part that will not survive a live copy.

"Live file copy safe: no" does not mean a snapshot is impossible. It means that copying the files while the service is running can produce a backup that restores into something broken, so the service needs either a proper export, a stop, or a crash-consistent filesystem snapshot. Backing up a running database explains the mechanics, and The backup blind spot shows how much of the index this affects.

This derivation is a first filter, not a substitute for the project's own backup documentation. Where a project publishes a supported export command, the profile says to use that instead.

The other vocabularies#

Identity support is recorded as one of: native OIDC, OIDC via plugin, trusted proxy header, LDAP only, auth proxy only, local accounts only, or not applicable. The distinction matters more than a yes or no, because forward auth and native OIDC give you very different things. Single sign-on for self-hosters covers why, and The single sign-on gap shows the distribution across the index.

Licence class is one of permissive, copyleft, source-available or proprietary. The exact SPDX identifier is recorded separately, because the class is the thing that matters for a decision and the identifier is the thing that matters for a check. Source-available covers licences that publish source but are not OSI-approved, such as sustainable-use and fair-use licences. We do not call those open source, because they are not, and blurring it does the reader no favours.

Maturity is one of mature, stable, young or slowed. A project is marked slowed when it still works but development has clearly decelerated, and legacy or archived projects stay in the index rather than disappearing, because people are still running them and still searching for them. Telling somebody their software is now unmaintained is more useful than pretending it never existed.

arm64 is true only where official arm64 builds or official multi-arch images exist. Distribution packages built by third parties do not count, and the profile says so where that distinction is what makes the answer "no".

What this index does not measure#

Being explicit about this matters as much as the rest of the page.

• We do not benchmark. No throughput numbers, no transcoding frame rates, no requests per second. If you see a performance figure here, it came from the project or it is arithmetic we showed our working for.
• We do not audit security. We record identity support and note published issues where a project documents them. That is not a security review.
• We do not count stars, downloads or forum activity. Popularity is not quality, and those numbers are easy to move.
• We do not score user interface quality. It is real and it matters, and it is not something we can apply consistently across 100 tools, so it lives in the prose instead of in a number.
• The index is not a census. It is a set of tools chosen for real-world use and genuine search demand. Aggregate percentages describe this index, not the whole self-hosted ecosystem, and the research pages say so every time they quote one.

The dataset#

The structured index is published under a Creative Commons Attribution licence:

• The full index as JSON
• The same data as CSV

Use it, cite it, argue with it. If your analysis contradicts ours, we would like to know, and corrections is how to tell us.

When we change our mind#

Software changes and so do verdicts. When a recommendation reverses, we change the page, move the review date, and say in the page what changed and why rather than quietly editing history. A page that has been silently rewritten is worth less than one that admits it used to say something else.