---
name: cfp-review-scorer
description: Reviews call-for-papers and call-for-speakers submissions for conferences, summits and user conferences. Applies an anchored 1 to 5 scoring rubric (relevance, originality, practical takeaways, speaker credibility, clarity) with reasons, flags thinly veiled product pitches, duplicates and missing info, balances tracks, levels and speaker mix, and produces reviewer packets plus accept, waitlist and decline recommendations with notes. Use for CFP review, abstract scoring, session selection or speaker selection.
license: MIT
metadata:
  author: eventmarketingsoftware.com
  version: "1.0"
  category: content-and-speakers
  source: https://eventmarketingsoftware.com/skills/cfp-review-scorer
---

# CFP Review Scorer

Pre-scores a pile of session submissions so the program committee spends its time on the close calls. Every score comes with a reason tied to the rubric anchors, every flag says why, and the shortlist respects your track slots. Humans make the final decisions; this skill makes them faster and more consistent.

## Intake

If the user has an `event-context.md` file, read it first and don't ask for anything it already answers (audience, sponsor list, themes).

Ask for whatever is missing in ONE message:

1. The submissions export (CSV or XLSX from your CFP or abstract management tool, a form export or a pasted table).
2. Tracks or themes, total session slots and slots per track, formats and lengths.
3. The rubric: use theirs if they have one, or the default in [references/scoring-rubric.md](references/scoring-rubric.md). Any criterion weights.
4. The audience: roles, experience level, what they come for.
5. Sponsor list and rules for vendor speakers (e.g. vendors only via sponsor slots, vendors must co-present with a customer).
6. Balance goals: max sessions per company, share of first-time speakers, level mix, any self-reported diversity goals.
7. Reviewers: how many, their names or initials, and known conflicts (their own company, close partners).

## Data sources (connectors optional)

- If a Google Sheets connector is available, read the submissions sheet and offer to write scores to a new tab. Never overwrite the source tab.
- Otherwise, work from an uploaded CSV or XLSX or a pasted table and return Markdown tables plus downloadable CSVs.
- Never require a paid service. Don't fetch speakers' websites or social profiles.

Sample file for testing: [assets/sample-cfp-submissions.csv](assets/sample-cfp-submissions.csv).

## Method

1. **Load and count.** Report rows in. Map columns to: submission ID, title, abstract, outcomes, format, track, level, speaker name, title, company, email, co-presenters, prior speaking, sponsor status. Ask only if a critical column (title, abstract, speaker) can't be mapped.
2. **Clean and flag before scoring.**
   - `Duplicate`: same or near-identical title or abstract from the same speaker or company. Keep the most complete or most recent, list the others.
   - `Multiple from speaker`: one person on several submissions. Fine, but only accept one unless the user allows more.
   - `Missing info`: no abstract, abstract under about 40 words, no outcomes when required, no speaker details.
   - `Over limit`: above the CFP's stated word limit.
   - `Vendor pitch`: product names, feature lists, "our platform", demo promised, or a vendor speaker with no customer and no sponsor slot. Quote the phrase that triggered it.
   - `Vendor not sponsor`: speaker's company sells to this audience and isn't on the sponsor list. Flag for the committee, don't auto-decline.
   - `Conflict`: a reviewer works at or partners with the speaker's company.
3. **Score each submission** on each criterion, 1 to 5, using the anchors. Write one short reason per criterion that points at the text ("Outcomes are specific: build a lead tier model"). Score only what's written. Don't reward famous companies or penalize unknown speakers beyond the credibility criterion.
4. **Compute the weighted total** (0 to 100): sum of (score / 5 x weight). Default weights in the rubric reference. If code execution is available, run `python3 scripts/cfp_shortlist.py` (see below) for totals, ties and the shortlist; otherwise do it in-chat and show the math for the top and borderline rows.
5. **Build the shortlist.** Fill each track's slots in score order, applying caps (max per company, one per speaker), then fill the waitlist (default 25% of slots) in score order. Note where balance goals changed the order and why. Ties: use the rubric's tiebreak question.
6. **Recommend.** Each submission gets `Accept`, `Waitlist`, `Decline` or `Committee review` (flags that need a human: vendor not sponsor, conflict, borderline score within 3 points of the cut line).
7. **Prepare reviewer packets.** Assign each `Committee review` and top-scoring submission to 3 reviewers (or as many as the user has), avoiding conflicts and balancing load. Use the packet format in [references/reviewer-packet.md](references/reviewer-packet.md).
8. **Draft notes.** Short, kind decline and waitlist notes per reason group (fit, capacity, vendor pitch), plus an acceptance note template. Drafts only.

### Script

`scripts/cfp_shortlist.py` takes a scored CSV and the slot plan and returns totals, duplicate flags, the shortlist and the waitlist. Python standard library only.

```
python3 scripts/cfp_shortlist.py scored.csv --slots "Demand Gen=4,Field Events=3,Ops and Data=3" --max-per-company 1 --waitlist-share 0.25 --out shortlist.csv
```

Scored CSV columns: `submission_id,title,track,speaker_email,company,relevance,originality,takeaways,credibility,clarity` (extra columns are kept). Several rows per submission (one per reviewer) are averaged. The script adds `total`, `rank_in_track`, `recommendation` (Accept, Waitlist, Decline), `borderline` and `flags` (duplicates, caps, tracks with no slots). You then move borderline rows and rows with human-judgment flags (vendor not sponsor, conflict) to `Committee review`. If code execution isn't available, follow the same steps in-chat.

Domain rules that matter:
- Anchored scales keep reviewers consistent. Every 1 and 5 needs a reason.
- At least 3 reviewers per final decision is a common standard; the pre-score is one input, not a reviewer.
- Never infer gender, ethnicity or other protected traits from names or photos. Use self-reported fields only if the CFP collected them and the user asks.
- Blind review: if the user wants it, hide speaker name and company in packets and score credibility separately.
- Vendor sessions aren't banned; they're judged on whether a competitor's customer would learn something.

## Output templates

**1. Intake summary:** rows in, rows scored, rows flagged by flag type, rows out (must equal rows in).

**2. Scored table** (also as CSV):

| ID | Title | Track | Format | Speaker, company | Relevance | Originality | Takeaways | Credibility | Clarity | Total | Flags | Recommendation | Reasons |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

**3. Shortlist by track:** slots, accepted, waitlist, and the cut-line score per track.

**4. Balance check:**

| Measure | Goal | Shortlist | Note |
|---|---|---|---|

(level mix, sessions per company, first-time speakers, customer vs vendor speakers, formats)

**5. Committee review list**, **6. Reviewer packets**, **7. Decline, waitlist and accept note drafts**.

## Validation checklist

- [ ] Rows in equals rows out; every row has a recommendation.
- [ ] Every flag quotes or cites what triggered it.
- [ ] Every 1 and every 5 has a reason that points at the submission text.
- [ ] Totals recompute correctly from scores and weights for a spot check of 3 rows.
- [ ] No track exceeds its slots; caps per company and per speaker hold.
- [ ] Borderline rows (within 3 points of the cut line) are in Committee review, not silently declined.
- [ ] No protected traits inferred.
- [ ] House style in notes: contractions, plain words, no dashes as punctuation.

## Approval gates

- The skill recommends; the committee decides. Label the output "Pre-score for committee review".
- Never send acceptance, waitlist or decline emails, update a CFP tool or write to a sheet without showing the content and getting a clear yes.

## Privacy

- Submissions contain personal data. Keep the file in the user's tools and don't repeat speaker emails in summaries beyond what's needed.
- Reviewer packets should include only what reviewers need. For blind review, remove names, companies and emails.
- Decline notes shouldn't share scores of other submissions.

## Related skills

session-abstract-writer (the speaker's side), agenda-builder (scheduling accepted sessions), speaker-management (onboarding accepted speakers).
