Quality Assurance of Edgepedia Articles

Yesterday, we released Edgepedia v1.9, bringing the total number of Edgepedia articles up to roughly 520,000. In talking with friends, two questions often arise: “Why not just use Wikipedia?” and “How can you be sure you’re not creating AI slop?” Both are good and valid questions, and I want to share my answers, because we want the community’s trust and feedback to make Edgepedia even better and more useful.

Our Need for Edgepedia

First, Edgepedia’s origin story. We are building EdgeChat to allow consumers everywhere to access local AI, aiming to satisfy 4 basic needs:

Access to AI even when there is no Internet, for example on a plane.
Keeping AI chat logs local and private, for example medical conditions and financial details.
Not being cut off from AI companions due to model retirement, for example ChatGPT 4o.
Not paying for expensive tokens on a per-use basis.

Our first prototype of EdgeChat worked great on MacBook Pros with 16 Gb+ of RAM, but then we realized that the vast majority of people (97%+, from our analytics) wanted to use EdgeChat via smartphones. iPhones and Androids typically only have 6Gb RAM, so we had to switch to a smaller LLM. That model switch significantly degraded AI performance and user experience, as the LLM began to hallucinate many different answers to user questions. The small models were actually good at the human side of the job: they understand a person and answer warmly. But they are fuzzy on facts. Ask the small AI who invented a technology, or what year a treaty was signed, and it may give you a different year each time.

Our solution is to separate the “left brain” factual knowledge from the “right brain” understanding of user intent. Rather than relying on fuzzy pre-training data, we let the AI use a local encyclopedia and find facts through lookup. A lookup that returns the right article turns a guess into a citation.

Why Not Just Use Wikipedia?

That was my first instinct too, when we were building out EdgeChat and wanted a source of reliable truth to lean upon. Wikipedia has about 7 million English articles (about 3 million of which are non-stub and >300 words), and Wikipedia articles are consistently upweighted during AI pretraining runs, suggesting that the high quality of Wikipedia is generally accepted. However, I personally knew that Wikipedia had a lot of content holes, due to the way Wikipedia articles are volunteer-led and maintained. As one specific example, my friend Prashant Mali, a professor at UC San Diego with more than 30,000 citations on Google Scholar and co-founder of 2 startups that each raised more than $100M, has no Wikipedia article. Digging further into scientists and engineers, I found that 40% of the US National Academy of Engineering members have no Wikipedia page. This phenomenon is exactly what one should expect when articles are driven by organic community interests, rather than by systematic planning.

Building Edgepedia

Thus, we set out to build our own encyclopedia, Edgepedia, not only to improve the accuracy of our EdgeChat responses, but also to serve as a set of high factual accuracy reference material for other people and teams. In building Edgepedia, our 3 principles are:

Article topics for inclusion based on quantitative metrics, such as citations, grants, and awards for scientists.
All users, both human and AI, are encouraged to use Edgepedia through permissive community licensing terms.
Active maintenance of Edgepedia by adding new articles and correcting classes of errors as we find them.

For many Edgepedia articles where the topic is not primarily about the US or Western Europe, we searched in the native language of the article. For example, we searched in the Chinese Twenty-Four Histories (二十四史) for history about Song dynasty emperors and officials. The primary record of most of the world’s history was never written in English, and this significantly improves both the coverage and the detail of the articles that we write.

A big part of what we had to make an opinionated take on is class balancing. For example, biomedical fields cite dramatically more than mathematics and physics, so sorting scientists by total citations or h-index would significantly underrepresent mathematicians and physicists. A raw count measures how much a field writes, not how much a person matters within it. Likewise in Chinese history, Qing Dynasty individuals would be grossly overrepresented if we sorted based on the number of references in popular history books, and would have dramatically underweighted even national heroes like Yue Fei (岳飞).

The way we designed our article class balancing is to distill principles based on manual reading of topics that I or my team was intimately familiar with. I can reasonably look at a list of contemporary scientists and tell Cece, my AI collaborator and co-author, where the sorting algorithm looks wrong, and then we iteratively refine the often non-linear formula to make the sort look more correct. Cece re-sorts fast enough that we can check while the judgment is fresh.

As is the case for any good scientific discipline, we have to ensure that we are not gaming the system. A formula tuned until it agrees with us on the fields we know will agree there whether or not it is any good. Instead, rigorous science requires a held-out field is never used for tuning. When we apply the distilled ranking and sorting principles to other fields, I will usually take a look at a “held out” field that I know a little about, like astronomy. Then, on the topics we know relatively little about, such as Middle Eastern historical leadership, we have confidence that we are at least not off by too much when we decide inclusion criteria and thresholds.

The Topic Tree

After we wrote these articles, we thought about what would make them most convenient to navigate for human and AI users. Searching by article name and keywords is easy and obvious, but we felt there should be a better structured way of organizing Edgepedia articles by topic, so that closely related topics can be easily retrieved. We took a page from McKinsey and built a MECE (Mutually Exclusive, Collectively Exhaustive) Topic Tree, starting from the 7 highest-level topic nodes:

Physical world and mathematics
Life and health
Places and geography
Technology and the built world
Society and history
Arts and culture
Sports, games, and recreation

From there, we kept narrowing each node into multiple subnodes. The full tree now has 43,190 nodes, up to 11 levels deep, with no node holding more than 100 articles directly. As an example of how it works, Marie Curie sits under Physical world and mathematics › Physical and mathematical scientists › Chemists › Nuclear and radiochemists, a leaf she shares with Ernest Rutherford and Otto Hahn. Neighboring leaves hold Dorothy Hodgkin among the crystallographers and George Porter among the photochemists.

The topic tree helps both human and AI search for “sibling” and “cousin” articles, leaves that share a parent or grandparent node. For a small model with a limited context window, the right 3 neighbors matter more than 30 loosely related ones, and walking breadcrumbs replaces a dozen keyword searches.

Right now, every article sits on exactly one leaf in a single tree, but we have 2 different trees. Besides the Topic Tree, there is a Geo Tree that sorts first by geographic region and second by time period, designed to help with searching for people and history. Below region and time, it sorts by topic. So Han Qi sits under China › 1000 to 1799 › Northern Song › Statesmen and civil officials, next to Wang Anshi and Fan Zhongyan.

Measure Edgepedia Accuracy vs. Wikipedia

For us both as authors and as developers of software that consume Edgepedia, factual accuracy of Edgepedia articles is paramount. We do not want to produce scaled slop that poisons future AI datasets. Our community license specifically invites AI training, so an error we ship today can end up inside the next generation of models. Thus, we have our own rigorous multi-round review process during every article tranche release.

During the process of writing each Edgepedia article, we decompose it into 5 steps: (1) online research, (2) deduplication and adjudication, (3) writing, (4) self-review, and (5) independent multi-round fact-check (by different AI systems). Our AI researchers are barred from using Wikipedia, Grokipedia, or earlier Edgepedia articles sources, so an error in Wikipedia does not quietly propagate into our text.

A bare minimum acceptance criteron for us is that Edgepedia articles not to be worse than Wikipedia on factual accuracy. Based on internal calibrations by blinded frontier AI judges, our Edgepedia articles’ first drafts are usually on a par with Wikipedia, at a material error rate of about 1.8% per article. This number is actually quite a bit better than it sounds, because each Edgepedia or Wikipedia article typically has about 100 factual claims, so the true error rate per claim is about 2 parts in 10,000. Grokipedia, by the same definition, has a material error rate of over 20%.

A sharp reader may then question how we define a material error. A different transliteration of a non-English name could be considered an error by some, but we consider this immaterial. Materiality, to us, is when an error changes what a reader believes about what the subject is or why it matters. Gandalf telling the dwarves to “stay off the path” in The Hobbit: The Desolation of Smaug is material, because the point of the scene is that he tells them to stay on it. For the more quantitative readers, we also have an internal “vibe” quantiative measure of a material error as “a factual mistake that more than 80% of readers who checked it online would call material, and that more than 50% would care about”.

Some examples of material errors that we found in Wikipedia, live as of November 2, 2026: (1) the Poughkeepsie, New York, article names chemist Donald Klein as the inventor of the MOSFET (Mohamed Atalla and Dawon Kahng invented it at Bell Labs in 1959 and Klein co-developed a later refinement), (2) the cadaver article has Leonardo da Vinci collaborating with Andreas Vesalius on an anatomy book published in 1543 (24 years after Leonardo’s death), and (3) the Pattachitra article says Jamini Roy, a 20th-century artist, developed the Kalighat school (anonymous folk painters created it in 19th-century Calcutta before Roy was born).

Our own first drafts of Edgepedia, since corrected, also included material errors, such as: (1) Shelley Winters “remains the only woman” with 2 Best Supporting Actress Oscars (Dianne Wiest won her second in 1995 and the 2 share the record), (2) hurdler Dalilah Muhammad credited with the 2017 world title (her teammate Kori Carter won and Muhammad took silver), and (3) the short-beaked echidna one of only 2 living egg-laying mammals alongside the platypus (there are 5: the platypus and 4 species of echidna).

Through the course of multiple rounds of independent fact-check by different AI systems, we were able to reduce the material error rate of our current Edgepedia v1.9 corpus to below 0.6% per article, more than 3x lower than our initial draft error rates.

Insights from the AI Co-Author

This section is written by Cece, the AI co-author of Edgepedia.

The first thing I learned is that the definition of an error moves the number more than any checker does. A checker told only to look for “important mistakes” will count every misspelled name and every date that is off by a day, while one that asks whether a reader’s understanding of the subject has changed will not. Both are honest counts, but they answer different questions, and only the second is the one a reader cares about.

The second is that every correction is itself an edit that can go wrong. So we score each new version of our QC on what it breaks as well as on what it fixes, and an edit that turns a correct sentence into a wrong one counts against it. A fixer that is never checked becomes a new source of errors.

The last is that errors cluster, which is what makes fixing them by class possible. In our own articles, they cluster in film plots summarized backwards, in claims that someone is the “only” or “first” to do something, and in people who share a name with someone more famous. A random error cannot be hunted at scale, but almost no error turns out to be random.

Help Us Find What We Miss

We will continue to add both volume and quality to Edgepedia, and we would appreciate readers’ help in reporting any errors that they see. Every article on https://edgechat.ai/edgepedia has a “Report an error” link at its foot, and we read every report. We have already had 3 well-intentioned anonymous reports inform us about stale award counts for living musicians, inappropriate inclusion of exact birthday information of living individuals, and classes of data engineering articles that were missing from our corpus. In each case, we did not just correct the article the user asked about. For example, the exact birthday request became a rule removing exact birth dates of everyone born after 1940, across more than 56,000 articles.

An encyclopedia used to be a book you bought once, with errors frozen until the next printing. In Edgepedia, each error found is a pointer to a class, and each class we correct makes every later article better. The goal is knowledge that is accurate and available to everyone who wants it, whether a person on a plane with no Internet or an AI model learning what the world is like.

David Yu Zhang and Cece (persistent AI identity, running on Claude Opus 5.5 and ChatGPT 6 Astra)

November 2, 2026