複合スコアリング

複合スコアリング

複雑な判断を最小単位のスコアに分解し、コード側で制御する重みで合成します。

複数の基準を同時に使って項目を順位付けしたいことがよくあります。複合スコアリングはこれを考えるための簡単な方法です。判断を独立した次元に分解し、それぞれを個別にスコアリングし、コード側で制御する重みで合成します。

例：履歴書のスクリーニング

エンジニア職の履歴書を処理しているとします。複数の基準で候補者を順位付けし、最終的に上位 X 名を次のレビューに回したいと考えています。

%%{init: {"fontFamily": "Inter, sans-serif", "flowchart": {"rankSpacing": 35, "wrappingWidth": 300, "subGraphTitleMargin": {"top": 12, "bottom": 36}}}}%%
flowchart LR
    resume["candidate resume"]

    subgraph req["TypeSafe evaluates questions<br/>in parallel"]
        direction TB
        py["<b>Score:</b> Python depth"]
        lead["<b>Score:</b> team leadership"]
        arch["<b>Score:</b> system design"]
        general["<b>Score:</b> generalist"]
        %% Invisible links stack the questions; they are answered in parallel.
        py ~~~ lead ~~~ arch ~~~ general
    end

    resume -- "one request<br/>resume + 4 questions" --> req
    req -- "one response<br/>4 score answers" --> normalize["<b>normalize scores to 0–1</b><br/>divide each by 4 in your code"]
    normalize --> ic["<b>senior IC weights</b><br/>40% Python + 10% leadership<br/>40% design + 10% generalist"]
    normalize --> em["<b>engineering manager weights</b><br/>15% Python + 40% leadership<br/>20% design + 25% generalist"]
    ic --> rank["rank candidates<br/>for each role"]
    em --> rank

ステップ 1：各次元を個別にスコアリングする

{
  "python_depth": {
    "type": "score",
    "instructions": "How much depth of python experience does this candidate have, based on the supplied resume?",
    "criteria": [
      "No Python experience mentioned",
      "Mentioned but no detail",
      "Used in projects, some specifics",
      "Primary language, multiple projects",
      "Deep expertise: architecture, performance, libraries"
    ]
  },
  "team_leadership": {
    "type": "score",
    "instructions": "How much experience does this candidate have managing or leading engineering teams?",
    "criteria": [
      "No management experience mentioned",
      "Informal mentorship or tech lead role",
      "Led a small team or project",
      "Managed a team with direct reports",
      "Managed multiple teams or an engineering org"
    ]
  },
  "system_design": {
    "type": "score",
    "instructions": "How much experience does this candidate have designing large-scale or distributed systems?",
    "criteria": [
      "No architecture work mentioned",
      "Contributed to design discussions",
      "Designed components of a larger system",
      "Owned architecture of a significant system",
      "Designed systems at scale across multiple domains"
    ]
  },
  "generalist": {
    "type": "score",
    "instructions": "How much evidence is there that this candidate picks up unfamiliar tools, roles, or domains outside their core specialty?",
    "criteria": [
      "Only one domain or role mentioned",
      "Some variety but within a narrow field",
      "Worked across a few different areas or tech stacks",
      "Regularly moved between domains, wore many hats",
      "Track record of ramping up in unfamiliar areas and delivering"
    ]
  }
}

ステップ 2：重みで合成する

各次元は 0–1 に正規化してから重み付けします。重みを使えば、個々のスコアが持つ細かなニュアンスを失うことなく、各次元の相対的な重要度を簡単に調整できます。

scoring.py

py      = response.answers["python_depth"].score / 4
lead    = response.answers["team_leadership"].score / 4
arch    = response.answers["system_design"].score / 4
general = response.answers["generalist"].score / 4

# Senior IC
ic_score = (0.40 * py) + (0.10 * lead) + (0.40 * arch) + (0.10 * general)

# Engineering Manager
em_score = (0.15 * py) + (0.40 * lead) + (0.20 * arch) + (0.25 * general)

これで複合スコアに基づいて候補者を順位付けできます。さらに重要なのは、最終スコアが具体的にどう計算されているかを可視化できる点です。上位の候補者が期待どおりでなければ、重みを調整して適切なバランスを見つけられます。