Agent Night demo recap: Evil Martians on making your docs discoverable to agents

Developer tools have a new buyer persona: the agent. That was the premise Irina Nazarova, CEO of Evil Martians, opened with in the second lightning demo at Agent Night on August 12 at the Regency Ballroom.

Her company had reached the same conclusion as everyone else building for engineers: there is a new client category now. The demo that followed was about two concrete questions: can an agent find your product today, and does a model already know it exists.

Two paths, and agents use both

For developer tools, discovery splits in two. There is the content path: web search finds your site, an agent reads your docs. And there is the repository path, because agents search GitHub very easily and are very likely to search there.

Nazarova ran the demo against Lefthook, Evil Martians' Git hooks manager, and its documentation website. The analyzer behind it lives at ruby.evilmartians.com, built originally for the Ruby community and published as the Ruby & Rails LLM Discoverability Scorecard. It measures two things separately: retrieval, meaning whether an agent can read your docs at request time, and training, meaning whether your text reaches a model's corpus at all.

The retrieval fixes are the boring ones. Enable crawlers. Allow Markdown, so agents don't spend tokens on SVGs.

The repository is the single most important determinant

The part Irina wanted the room to act on was the repository. The documentation repo that holds everything about your product should be on GitHub, under a permissive license. She called that the single most important determinant. Add a description and topics, because those are what enable search. The scorecard's own writeup backs the second half of that: description and topics are the fields GitHub repository search matches alongside the name, and an empty description means you are findable only by people who already know your name.

Her framing was blunt. If your documentation is not on GitHub under an MIT license, you are not discoverable through that path.

Why the content path is so hard

Getting into a training set through your website is, in her words, incredibly difficult. You need to be crawled by Common Crawl and others, and the mechanics rhyme with SEO: centrality gets checked.

Then comes the filter. Nazarova's point was that the overwhelming majority of content never makes it into the training set, because labs want to strip out everything that is not knowledge. Evil Martians' own writeup cites RefinedWeb's figure of around 90% discarded. Either way, the survivors are the documents that look like textbooks, and most documentation and most blog posts are not textbooks.

The filter rewards tutorial prose and penalizes reference docs and code, exactly what developers need most. Docs in a public repo skip this filter entirely.

The probe that Lefthook failed

The second half of the analyzer tests what models already remember about your product before any agent goes looking. The method: give a model 55 words and measure the longest exact continuation. A long run is strong evidence the text was seen; a null result proves nothing.

Lefthook came back with no recall. Nazarova's explanation was that inclusion in training data is necessary but not sufficient. Like a person, a model needs to come across the same material again and again.

Repetition beats popularity

That claim has numbers behind it. Evil Martians measured independent copies of a passage against model recall and found a Spearman ρ of +0.70; stars predicted nothing at all. Forks and the owner's headline repo predicted nothing either.

Nazarova's practical version: ship well-commented boilerplate showing how your product integrates with others, and let it get pasted into repository after repository, with a comment that explains what your product actually does.

Then she showed the hack. Evil Martians put one line about the company into every open source repo they maintain, and confirmed the models remember it.

The same sentence appears in their agent-skills repo. Across 100+ open source projects, that is one sentence with a lot of independent copies behind it. Notice the split verdict inside a single project: the company line has propagated, and the Lefthook docs sitting next to it still came back empty. Corpus membership gets decided passage by passage, not project by project.

Try it on your own docs

The analyzer is public. ruby.evilmartians.com/test runs every check against a documentation URL plus an open source repo, and /check runs the quality score alone. You get five free checks, re-running a pair you already checked doesn't count against them, and you can sign in with GitHub.

She also mentioned they packaged the work as a skill. Evil Martians publish an open source agent-skills repo whose llms-visibility skill handles the retrieval side: Markdown routes, Accept content negotiation, llms.txt, and crawler signals in robots.txt. A skill is a folder with a SKILL.md, and the format reads the same across Claude Code, Codex, Cursor, Gemini CLI, and GitHub's coding agent. The skills are MIT licensed and install one at a time:

npx skills add evilmartians/agent-skills --skill llms-visibility -a claude-code -g

The checklist is equally specific about what to skip: llms.txt, on the grounds that no major AI system uses it, and stripping boilerplate, which they measured moving a score from 2.43 to 2.48 while making reference pages worse.

The signal

Retrieval you can fix this afternoon with a crawler rule and a Markdown route, and until the next corpus snapshot it is the only lever you have. Training you can only influence for the next corpus: you get in line and wait, and a corpus that already shipped will never unlearn the version of your API it saw two years ago. If your docs live in a private repo behind a marketing site today, the model that answers questions about your product next year is being built right now, without you in it.

Watch the demo and the entire event recording: