The quality score of AI resources

Each skill, agent and rule gets a score out of 100, calculated from public good practices. You see which checks pass, which fail, and what to change in the repository.

Before you start

Compass is on and AI resources is on. Astrolabe creates three compasses for you: AI skills, AI agents and AI rules. A runbook has no score.

🧭 Read the score

1. In the left sidebar, click AI resources. A card shows Score: 100/100, or Score: 40/100.

2. Click the name of a resource, then click the tab Compass. A gauge shows the score, and under Assessments each check has a green check (passes) or a red cross (fails).

3. Click a check to open it. It shows Result (Passed or Failed), what Astrolabe Read in the file (a field in quotes, (empty) when it is missing, or a count of lines, characters or bytes), what it Expected when the check has a limit (for example at most 200 lines), what to change when it fails, Fix it in the source to open the file, the Source to read, and its weight. For the agent helper, Model declared says:

   • Result: Failed
   • Read in model: (empty)
   • Fix it in the source
   • To do: Set model (sonnet, opus, haiku or inherit), so its cost and behaviour are known.
   • Astrolabe's choice, inspired by the source — The page recommends it; Astrolabe makes it a check.
   • weight: 25 %

4. Click Source to open the page the check comes from.

The checks that concern the file share the score equally: a Claude Code agent has four checks of 25 % each.

🏷️ Written by the source, or Astrolabe's choice

Under each check, one of these labels says where it comes from, followed by what it means:

Label	Meaning
Written by the source	The page behind Source states the rule.
Astrolabe's choice, inspired by the source	The page does not require it; Astrolabe asks for it because it helps people find and trust a resource.
Company rule	Your company wrote it in Astrolabe. Company page opens the page your company gave for it.

✅ The checks of a skill

Check	What to do when it fails	Label
Name follows the standard	Set name in SKILL.md: 1 to 64 lowercase letters, digits and hyphens (pdf-processing).	Written by the source
Name matches the folder	Set name in SKILL.md to the name of its folder.	Written by the source
Has a description	Write a description in SKILL.md, 1,024 characters at most.	Written by the source
No XML tags	Remove the tags from the name and the description.	Written by the source
Under 500 lines	Move details out of SKILL.md into the files it links to: keep it under 500 lines.	Written by the source
Says when to use it	Say in the description when to use the skill (« Use when… »).	Written by the source
Specific name	Rename the skill: helper, utils, tools, documents, data or files say nothing of what it does.	Written by the source
Forward-slash paths	Write the paths of the skill with /, never \.	Written by the source
License declared	Set license in SKILL.md, so people know whether they may reuse it.	Astrolabe's choice
Tagged	Add tags to the front matter, or to the repository in the catalog, so people find it in the store.	Astrolabe's choice

Sources: the Agent Skills specification, the best practices of Anthropic, the skills of Claude Code and the guide for companies of Anthropic.

✅ The checks of an agent

Check	What to do when it fails	Label
Has a name and a description	Set name and description in the front matter of the agent.	Written by the source
Says when to use it	Say in the description when to use the agent (« Use proactively after… »).	Written by the source
Tools restricted	List the tools the agent needs in tools:: without it, it gets all of them.	Written by the source
Model declared	Set model (sonnet, opus, haiku or inherit), so its cost and behaviour are known.	Astrolabe's choice
Within Copilot's limit	Shorten the prompt of the agent: Copilot reads 30,000 characters at most.	Written by the source

Sources: the sub-agents of Claude Code and the custom agents of GitHub Copilot.

✅ The checks of a rule

Check	What to do when it fails	Label
CLAUDE.md under 200 lines	Split CLAUDE.md into .claude/rules files: keep it under 200 lines.	Written by the source
Cursor rule under 500 lines	Split the rule: keep it under 500 lines.	Written by the source
Cursor rule scoped	Set alwaysApply, a description or globs, so Cursor knows when to apply the rule.	Astrolabe's choice
applyTo set	Set applyTo with the files the instructions apply to.	Written by the source
AGENTS.md under 32 KiB	Shorten AGENTS.md: Codex reads 32 KiB of it at most.	Written by the source
Copilot instructions under 4,000 characters	Shorten the instructions, or put the review rules first: Copilot code review reads the first 4,000 characters.	Written by the source

A check that does not concern the file is left out of its score: a Cursor rule is scored on the two Cursor checks only. The tab Compass names them above the list, for example 1 assessment does not concern this file: Within Copilot's limit.

No check concerns a GEMINI.md nor a rule in .claude/rules/. Their card says Score: no check for this kind of file, and their tab Compass says No check of this compass is about this kind of file: there is nothing to score.

Sources: memory of Claude Code, rules of Cursor, instructions of GitHub Copilot, code review of GitHub Copilot and AGENTS.md of Codex.

🏢 Add the rules of your company

Your company has its own rules: a description in French, a tag on every resource, no password in a file. Write each one in Astrolabe, and every resource it concerns is scored on it.

You need the right to create and change Compass assessments.

Write the rule

1. In the left sidebar, click Administration, then Compass, then the tab Assessments.

2. Click Create Assessment. A window opens.

3. In Name, write the name of the rule, for example Description in French.

4. In Description, write what to do when it fails, for example Write the description of the agent in French. People read it under the failed check.

5. In Datapoint, choose the kind of rule, for example Company rule: frontmatter field matches.

6. In Passes when the check is, choose True.

7. Fill in the settings under Parameters. For a description in French:

   • Frontmatter field: description
   • Regular expression: [àâçéèêëîïôûù]
   • Only for these formats (optional): claude-code/agent

8. In Link to the company's page (optional), paste the address of your internal page about this rule, for example https://intranet.example.com/ai-rules.

9. Click Save. The rule appears in the list, in the block AI resources, and a message says Assessment created. It counts in a score once added to a compass: tab Compasses, Edit, Add Assessment.

Add it to a compass

1. Click the tab Compasses. On the card AI agents, click Edit.

2. Under Assessments, click Add Assessment. Tick your rule, then click Save Selected Assessments.

3. Click Save.

✅ That's it! The next time a resource is read, its tab Compass shows your rule. When it fails, it shows what it read, what it Expected (for example description matching “[àâçéèêëîïôûù]”), Fix it in the source, your description after To do:, and Company rule, with Company page.

The kinds of rule

Datapoint	Settings	Passes when
Company rule: frontmatter field set	Frontmatter field	The front matter has this field.
Company rule: frontmatter field equals	Frontmatter field, Value	The field has exactly this value.
Company rule: frontmatter field matches	Frontmatter field, Regular expression	The field matches the expression.
Company rule: tag required	Tag	The resource has this tag.
Company rule: at most N lines	Maximum lines	The file has at most this many lines.
Company rule: at most N bytes	Maximum bytes	The file weighs at most this many bytes.
Company rule: format allowed	Allowed formats	The file is in one of these formats.
Company rule: file contains	Regular expression	A line of the file matches the expression.
Company rule: file does not contain	Regular expression	No line of the file matches the expression.

Frontmatter field of the two rules that read a value lists the fields Astrolabe keeps: name, description, model, tools, license, applyTo, globs, alwaysApply.

Every rule also takes Only for these formats (optional): a list separated by commas, for example agent-skills/SKILL.md, claude-md. On a file of another format, the rule is left out of the score.

Format	Files
agent-skills/SKILL.md	a skill (SKILL.md)
claude-code/agent	a Claude Code agent
copilot/agent	a GitHub Copilot agent (*.agent.md)
claude-md	CLAUDE.md
agents-md	AGENTS.md
gemini-md	GEMINI.md
copilot/instructions	.github/copilot-instructions.md
copilot/path-instructions	*.instructions.md
cursor/rule	a Cursor rule (.mdc)
claude-code/rule	a rule in .claude/rules/

tip

A regular expression follows the RE2 syntax. (?i) at the start ignores the case: (?i)confidential.

Problems you may meet

What the window says under a field	What to do
Not a regular expression Astrolabe can run (RE2: no look-around, no back-reference, 500 characters at most).	Remove (?=…), (?!…) and \1 from the expression, or shorten it.
Maximum lines must be a whole number, 0 or more.	Write a number without sign nor decimal, for example 300.
Unknown format: …	Write the formats exactly as in the table above.
The link must start with https:// or http://.	Paste the whole address of the page.

🔄 See the score go up

1. Correct the file in Git and push it to the default branch.

2. In the left sidebar, click Catalog, click the component of the repository, then click the tab AI resources.

3. Click Read the repository again. When it is done, the message Repository read again, score updated. appears at the bottom of the screen.

4. Open the resource, then its tab Compass: the check you corrected has a green check.

✅ That's it!

tip

Without Read the repository again, Compass scores the AI resources every 6 hours.

If you see Score: not measured or Not scored yet instead of a number, see Troubleshooting.