The same member cohort is evaluated under original and proposed membership rules before release.
Proposed teaching extension; NOVA does not currently ship this content-complete cohort impact tool.

Raise a synthetic membership condition from two direct registrations to three, and one member in the lab loses eligibility. The other two members keep their previous decisions. The program prints the affected member, the benefit and the reason, while a saved decision continues to replay under its original rules after the database is reopened.

That is the deliverable in this tutorial. You get an offline Python program that compares two rule snapshots and stores a decision with its exact input facts. The example contains no token pricing, payments or reward calculations. Its thresholds and members are invented test fixtures, not NOVA operating parameters or customer data.

I checked NOVA Membership's rule service and snapshot mapper before choosing this topic. The service supports draft creation, validation, independent review, scheduling and activation. Its simulation method records input facts and result traces. The current snapshot stores identifiers and configuration counts, and its hash derives from identifiers and version fields. A content-complete snapshot and cohort comparison would be additional engineering work. The downloadable code lets a team inspect that proposal before including it in a product scope.

Download the membership rule rollout lab, including the complete implementation, 14 tests and run instructions.

Make the change measurable

The lab evaluates an illustrative member_eligibility benefit. A member needs a Foundation pass and a minimum direct-registration count. Two frozen snapshots change only the threshold. The fixture contains a member with two independent pass types, one with a higher count, and one without any pass.

from rollout import fixture, impact

old, new, members = fixture()
report = impact(old, new, members)
assert len(report["changes"]) == 1
assert report["changes"][0]["member"] == "member-a"
assert report["changes"][0]["after"]["reason"] == "direct_requirement"

The pass types remain a collection. NOVA's product materials describe three independent tiers that can coexist, so reducing them to a single highest-tier field would discard information. The sample rule language checks one required pass per benefit. It deliberately stays small enough that a reviewer can inspect every supported condition.

The changed member's result moves from eligible to direct_requirement. A member without the required pass receives missing_pass. These reasons let a reviewer distinguish a rule threshold change from missing ownership. The example performs eligibility calculations only. Connecting a result to protected content or an event admission system would require a separate enforcement integration.

Keep the rule bytes that produced the answer

A version label helps a person identify a release. Reproduction also requires the conditions that were used. freeze validates the small rule schema and serializes the complete object with deterministic key order. It returns a string detached from the caller's dictionary.

def canonical(value):
    return json.dumps(
        value, sort_keys=True, separators=(",", ":"), ensure_ascii=True
    )


def digest(text):
    return sha256(text.encode("utf-8")).hexdigest()

Reordering dictionary keys keeps the same serialized snapshot. Changing a threshold changes the digest. Mutating the original dictionary after freeze leaves the saved string intact. The tests check all three behaviors.

This encoding is a contract for this Python lab's limited schema. It does not claim interoperability with every JSON implementation. The validator accepts nonnegative integer thresholds and explicitly rejects booleans and floating-point thresholds. Teams extending the schema must decide how new numeric types and string normalization affect their encoding before relying on cross-language hashes.

The digest helps detect accidental changes when the expected digest is retained. Someone with write access to the database can rewrite both content and digest. The lab supplies neither authenticated storage nor signatures, and its snapshot table is not an external audit service. A production record's authority still depends on access controls and retention arrangements.

Run the tests from the download

Extract the ZIP into an empty directory with Python available. The program uses only the standard library and targets Python 3.10 or newer. The included test log names the runtime used for this draft.

python3 -m unittest -v
python3 rollout.py

The first command runs 14 tests. The second prints a before-and-after report containing one changed member, followed by the original pinned decision. The final ZIP was extracted into a fresh temporary directory before both commands were executed. The product's own Java services were not started or retested for this article.

Try changing the proposed threshold to one. Then remove the benefit entirely and add a different benefit. The report compares the union of benefit codes and represents a missing definition as undefined_benefit. This prevents an empty new result from silently hiding a removed entitlement. The code also rejects duplicate member identifiers in the cohort because otherwise a report could count the same person twice.

Evaluate the same member facts under original and proposed rules, compare eligibility reasons, and retain original rules for historical replay.
Proposed teaching extension; NOVA does not currently ship this content-complete cohort impact tool.

Pin historical decisions before a retry

A delayed event can cross a release boundary. NOVA's inspected purchase processing selects an active snapshot during processing, which makes the version-selection moment a concrete design question. A project must decide whether a decision uses rules at purchase time, confirmation time or another agreed event.

This lab makes that choice explicit at its function boundary. The caller supplies the snapshot. Decisions.record saves the full rules, normalized member facts and an evaluator identifier under a synthetic event ID. Reusing that ID with identical inputs returns the same saved answer. Reusing it with different rules or facts raises an error.

from rollout import Decisions, fixture

old, new, members = fixture()
store = Decisions("decisions.sqlite")
try:
    original = store.record("purchase-demo", old, members[0])
    assert store.replay("purchase-demo") == original
finally:
    store.close()

The database operation inserts only if the event ID is absent, then compares the persisted request and result with the proposed values in the same transaction. An unexpected retry therefore cannot quietly replace the earlier decision. The reopening test closes the connection, creates a new connection to the same temporary database and reproduces the saved result from the stored facts and rules.

Replay also checks the evaluator identifier. A rule snapshot cannot reproduce an old answer if the algorithm's meaning changes underneath it. This lab supports only eligibility-v1. Future versions would need to retain the corresponding evaluator or provide an explicitly reviewed migration. Replaying the saved result does not reverse a service already delivered or automatically restore eligibility under current rules.

Use the report before scheduling a release

NOVA's existing review guard requires the reviewer to differ from the rule creator. The inspected service also checks stored status when transitioning a release and records simulation traces. A cohort impact report could give that review a more useful input, especially when a threshold change affects existing members. The report remains an educational extension here; it has not been integrated into the product's review screen or activation transaction.

For a buyer, request an example built from the conditions your community actually uses. Agree on the facts a decision needs and the moment those facts are captured. Then ask what happens to members who lose eligibility under the candidate version. That policy determines whether the report blocks release, triggers a correction or accompanies a planned transition.

The earlier NOVA rule-versioning article explains the current release sequence. The NOVA Membership product page describes the membership engineering available for customization. This lab adds a concrete acceptance exercise. Change a condition, inspect the affected cohort, close the database and replay an earlier decision from its retained inputs.