Building a PCPartPicker-Style Compatibility Engine for AI Systems

PickYour.AI is an attempt to do for AI systems what PC parts-list builders do for desktop computers: pick a workload and a deployment mode, then choose a model, the infrastructure to run it, optional retrieval, agents and fine-tuning, and operations tooling, with every choice checked against the others as you go. It is built in the open by the Pick-Your-AI GitHub organization as two repositories: a research repository that defines a shared, versioned data contract and does the technical and market research, and a static website that consumes that contract.

It exists because a modern AI stack is no longer one product. A model choice quietly constrains the GPU, the inference engine, the context budget, the embedding dimensions in your vector index and the monthly bill, and those constraints are usually discovered late. The question is whether those relationships can be written down once, as data, and checked mechanically.

Status: this is a preview and research prototype, not a recommendation service. Every component, price and compatibility result in the current website catalog is synthetic demonstration data. Do not use it to make purchasing or deployment decisions. PCPartPicker is a third-party trademark; I use the name only as an analogy for the interaction pattern, and PickYour.AI has no affiliation with PCPartPicker.

Problem

On a PC, compatibility is mostly physical. In an AI stack the constraints cross layers and run in both directions. The contract’s dependency notes list familiar examples:

• Switching from a hosted API to self-hosting suddenly requires weight access, a host, compute, an inference engine and a licence check.
• Changing the model can invalidate the GPU, the engine, the context policy, any fine-tuning plan, the retrieval setup and the cost estimate at once.
• Changing the embedding model forces a check of vector dimensions and distance metrics, and may mean rebuilding an index.
• Some checks are not pairwise at all. Whether a model fits on a GPU depends on model, quantization, batch size, context length, tensor parallelism, KV-cache data type and the interconnect, evaluated together.

The contract’s own wording is that the configurator is “a constraint graph, not a linear wizard”. The website still shows tiers in order, because that is how people think, but validity is computed across every active selection no matter which tier changed.

Solution

The component model: 29 kinds in seven tiers

The shared contract (version 1.0.0, under contracts/v1/ in the research repository) defines 29 component kinds grouped into tiers T0 to T6:

Several tiers are optional on purpose: a pure API deployment can omit self-hosted infrastructure, and RAG can be keyword-only. The T5 research checked this with the reference evaluator: six baseline rules triggered by training selections report skipped when training is removed or disabled, and no other rule outcome changes.

Architecture

One versioned contract shared by research and the website

The part of this design I would reuse anywhere is the contract boundary. The research repository owns component kinds, enums, schemas, the dependency graph and the ruleset. The website vendors contracts/v1/ byte-for-byte at a pinned commit, records a SHA-256 hash for every file in contracts/PROVENANCE.json, and fails its checks if anyone edits the vendored copy or it drifts from upstream. TypeScript types and Ajv validators are generated from the schemas, ahead of time and without eval, so the site’s content security policy can stay at script-src 'self'.

The website does not own compatibility policy. Its evaluator is a TypeScript port of the Python reference interpreter, and a parity test requires identical reports from both on seven scenarios, including self-hosting without capacity evidence and a RAG dimension mismatch. Rule changes go through a review issue in the research repository and become a new contract version; nobody patches the baseline in place.

Website architecture

The website is a static single-page application (React 19, strict TypeScript, Vite, React Router, TanStack Query). The host has no Node runtime, so the build must be fully static, and a script checks the output against a written deployment contract.

Data access goes through provider interfaces. The default VITE_DATA_MODE=fixture uses the synthetic catalog, pricebook and starter builds; http mode targets the contract API in api/openapi.yaml and validates every response against the schemas. Share links carry the build in the URL fragment, which is not sent to a server.

Implementation

Cross-component compatibility as data

The contract ships a baseline dependency graph with 59 directed edges between kinds and a baseline ruleset with 45 rules. Every graph edge has an activation condition (always, self_hosted, non_api, rag_enabled, agents_enabled, fine_tuning_enabled, if_selected or production_only), so that turning on RAG or switching to self-hosting brings whole branches into play.

Rules are written in a small data-only expression language. The contract is explicit: no eval, no executable strings, no SQL fragments. Here is the assertion from the rule that checks embedding and vector-index dimensions:

# contracts/v1/baseline/core-ruleset.json (rule.embedding.dimension, excerpt)
      "assert": {
        "op": "eq",
        "left": {
          "ref": {
            "scope": "selection",
            "kind": "embedding_model",
            "role": "primary",
            "pointer": "/settings/output_dimensions"
          }
        },
        "right": {
          "ref": {
            "scope": "selection",
            "kind": "vector_database",
            "role": "primary",
            "pointer": "/settings/index_dimension"
          }
        }
      },
      "on_unknown": "block",
      "message": "Embedding and vector index dimensions do not match.",

Unknown is never compatible

The most important design decision is that results are three-valued. Each rule ends as pass, fail, unknown or skipped, and severity (info, warning, error) is a separate field. A missing price, an uncalibrated memory estimate or an unresolved version produces unknown, not a quiet pass. The reference interpreter implements the logic in a few lines:

# contracts/v1/reference/evaluate.py
def boolean_and(args):
    if False in args:return False
    if UNKNOWN in args:return UNKNOWN
    return True


def boolean_or(args):
    if True in args:return True
    if UNKNOWN in args:return UNKNOWN
    return False

A rule whose applies_when condition is false is skipped, which means “not relevant”, not “compatible”. If the condition itself is unknown, the rule is unknown. Even when every rule passes, the reference evaluator reports deployment readiness as needs_review, because compatibility alone says nothing about quality evaluations, approvals or production readiness. I ran it against the contract’s synthetic fixtures and got the documented result: 41 passed, 4 skipped, 0 failed, 0 unknown, readiness needs_review.

Changes invalidate in both directions

When a selection changes, the build gets a new immutable revision, and the kinds connected to the change through any active edge, in either direction, are marked stale. The website’s implementation is a short loop over the graph:

# src/domain/invalidation.ts (pick-your-ai-website)
  for (const edge of graph.edges) {
    const ends: [ComponentKind, ComponentKind] = [edge.dependent_kind, edge.depends_on_kind];
    const activeBefore = edgeActive(edge.activation, before, ends);
    const activeAfter = edgeActive(edge.activation, after, ends);
    const active = activeBefore !== false || activeAfter !== false;
    if (!active) continue;
    const touchesDependent = changed.has(edge.dependent_kind);
    const touchesDependency = changed.has(edge.depends_on_kind);
    if (!touchesDependent && !touchesDependency) continue;
    // Both ends of an active edge participate.
    for (const k of ends) {
      if (!changed.has(k)) affected.add(k);
    }
    via.push(edge);
  }

Checking activation before and after the change catches a mode switch that turns a whole branch on or off, and an edge with unknown activation counts as active. The graph can contain cycles, so this is adjacency traversal, not a topological sort.

Model providers and deployment modes

The T1 research separates the model publisher, the distributor, the runtime operator and the offering you select. Billing channel, cloud branding or the presence of an API do not decide the deployment mode, and multiple providers alone do not make a build “hybrid”; mode follows who operates the runtime.

GPUs, infrastructure and inference engines

The T2 research treats advertised memory, provisioned allocation and usable runtime memory as three distinct facts, and recommends evaluating model, GPU and engine jointly using exact revisions, data types, quantization kernels, drivers and topology. It proposes engine catalog records for specific releases of vLLM, llama.cpp and TGI rather than a generic “supports X” flag, and it notes that arithmetic precision support does not mean an arbitrary quantized checkpoint will run.

It includes a dense-model KV-cache memory formula with a script for worked examples, labelled as an uncalibrated estimate that does arithmetic only. The reference evaluator follows the same line: runtime memory is “deliberately never guessed from weight size”.

Context windows and retrieval

For context windows, the T1 research says to count the complete serialized request for the selected route, including tool schemas, history, retrieved content and memory, and to reserve output tokens once under the applicable caps. A character ratio or a subtotal of one fragment does not establish that a full request fits.

For retrieval, the T3 decision is conservative: omit retrieval when it is not needed, and for document search compare keyword/BM25 first. Dense, hybrid and graph retrieval need a justified workload and their own evidence; no vector service is required by default.

Fine-tuning dependencies

T5 asks whether the selected model can and should be customized, and what the resulting artifact needs downstream. The baseline rules check base-model lineage, dataset governance, whether the training framework supports the method and backend, and whether the produced artifact can be served by the chosen inference engine. The research summary records that its four training-framework catalog proposals are all marked unverified.

Synthetic fixture data

The synthetic data is labelled in the code as well as the UI. Every fixture item is tagged synthetic and carries this provenance note:

# src/data/fixture/catalog.ts (pick-your-ai-website)
const PROVENANCE_NOTE =
  'SYNTHETIC DEMO FIXTURE for the PickYour.AI preview. Not a real product, vendor specification, price or measurement.';

Evidence

Everything in this section was captured on 2026-10-21 at website commit f47f3eb and research commit da3d320.

What I ran

Everything ran locally against synthetic fixtures, with no credentials or third-party calls. In the website, after npm ci --ignore-scripts, the deployment-script tests reported 115 passed and Vitest reported 474 passed in 45 files, including the Python parity tests; tsc -b, vite build and the check-dist script succeeded. In the research repository, the 21 contract tests passed and the reference evaluator reproduced its documented fixture result. I did not run the Playwright end-to-end and accessibility tests, lint, Lighthouse or the dependency audit.

The builder in fixture mode

I built the site locally at the website commit listed below and loaded the “Self-hosted open model on one GPU (demo)” starter build. All names in the screenshot, such as “Demo Open 8B” and “Demo C24 GPU”, are fictional fixtures.

A few interaction decisions stood out to me in the code and changelog:

• A first visit opens an empty build that reads Incomplete, not a prefilled stack, because a preselected stack would look like a recommendation. Starter stacks are an explicit choice.
• The status strip carries the worst outcome. In the screenshot, all nine required slots are filled, but the build shows Unknowns, 2 checks unknown with the note “Unknown is not a pass”.
• Per-component prices are never shown as $0. A component with no applicable rate is Unknown, a partly priced one is a lower bound, and “included” depends on the build policy.
• Evidence labels show recorded provenance (“Recorded: unverified”), and findings show outcome and severity as words, with an explicit “what was not checked” note.

That is the behavior I care about most. It would be easy to show a green tick for a 24 GB demo GPU. The rules require a calibrated per-device runtime estimate, none exists, so the honest answer is unknown, at error severity, which blocks deployment readiness.

Technical and market research

Besides the T0 to T6 technical workstreams summarized above, which mostly define interfaces and evidence requirements rather than recommend products, the research repository runs a market and customer research program.

The market synthesis is unusually blunt, and I think it is worth repeating as the repository states it. At its fixed evidence cutoff, the program decision was to narrow the work to a bounded comparison of whether a reconstructable “decision record” improves one coherent comparison over the best existing substitute and a plain README or manifest. It rejected a broad T0 to T6 builder or commercial expansion “at this checkpoint”, and it also declined a blanket stop. The reasons it gives:

• Existing tools already provide basic advice and conflict detection, so exclusivity of the basic workflow fails; whether a complete decision record is an advantage is unknown.
• Unrestricted commercial reuse of the candidate data sources is not established; all ten investigated integrations remain unapproved.
• Published benchmark results could not be pinned to exact models, harnesses and settings, so no model ranking is supported.
• Whole-runtime memory calibration is unknown, and zero real resource estimates had been admitted.
• Paid demand is unknown; no buyer or transaction has been observed.

The competitive feature matrix at that cutoff had 1,584 cells across 33 entities: 121 documented, 1,463 unknown, zero exercised. The synthesis warns that this is documentary coverage, not a product score. It records what is not known as carefully as what is.

Limitations

• The catalog is synthetic. Every component, price and compatibility result in the website preview is fictional demonstration data. It is not a quote, a vendor specification or a benchmark, and it must not be used for purchasing or deployment decisions.
• The rules are starting proposals. The contract describes the 59 edges and 45 rules as proposals that must be validated for a target workload. Passing the fixture tests does not validate real vendor facts, GPU capacity, pricing or production readiness.
• The evaluator is not a sizing tool. The reference interpreter and its port are explicitly not calibrated VRAM estimators, benchmark engines or deployment gates. The T2 KV-cache formula is labelled an uncalibrated estimate.
• Compatibility is not readiness. A valid evaluation means the configured rules passed for one locked revision and its evidence. Quality, safety, approvals and rollback evidence are separate.
• The market research narrowed the scope. At its stated cutoff, the program rejected broad expansion and commercial launch claims; data rights, comparable benchmarks, calibrated sizing and paid demand were all unestablished.
• Builds are local-only. Saved builds live in the browser; there are no hosted accounts or shared persistence.
• Many research items are still open, including contract extensions proposed by the workstreams that are not yet in a new contract version.

No affiliation: PCPartPicker is a trademark of its owner, used here only as an analogy. PickYour.AI is not affiliated with or endorsed by PCPartPicker; the research repository says the project develops its own design, wording, data model and implementation.

Resources

• Pick-Your-AI/pick-your-ai-website: MIT License, commit f47f3eb
• Pick-Your-AI/pick-your-ai-research: MIT License, commit da3d320

If you are trying to turn an idea like this into a working prototype, or need to decide which parts of an AI stack actually fit your data and constraints, CompleteTech offers Prototyping and MVP Development and AI-Driven Analytics.

PickYour.AI is a team project of the Pick-Your-AI GitHub organization. The research workstreams list lead assignees @tynamite, @zwright8, @Mattycodes69, @zabyxc01, @CompleteDotTech, @appletechie and @PUSHINGSQUARES, and recent commit history includes contributions from Timothy Wayne Gregg, Matthew Stewart, karutoil and MrTrenchTrucker.