``` ### Iterating to ~30% coverage (refactors and bugfixes) After that, we started iterating, slowly climbing to about 30% pass rate. This involved a lot of refactoring and fixing small bugs. ### Refactoring into per-tag handlers Once I could see the shape of the problem, I decided I liked a handler-based structure, where each tag gets its own handler. Modular structure ftw! I asked the agent to refactor and it did. ```python class TagHandler: """Base class for all tag handlers.""" def handle_start(self, context, token): pass class UnifiedCommentHandler(TagHandler): """Handles comments in all states.""" def handle_start(self, context, token): context.insert_comment(token.data) ``` ### Reaching 100% test coverage (with better models) From there, we continued iterating to 100% test coverage. This took a long time, and the [Claude Sonnet 3.7](https://www.anthropic.com/news/claude-3-7-sonnet) release was the reason we got anywhere at all. ### Benchmarking and discovering we were 3x slower With correctness handled, I set up a [benchmark](https://github.com/EmilStenstrom/justhtml/blob/master/benchmarks/performance.py) to test how fast my parser was. I saw that I was 3x slower than `html5lib`, which is already considered slow. ### Rewriting the tokenizer in Rust (and barely matching `html5lib`) So I tried the obvious next move: I let an agent rewrite the tokenizer in Rust to speed things up (note: I don't know Rust). It worked, and the speed barely passed `html5lib`. It created a whole `rust_tokenizer` crate with 690 lines of Rust code in `lib.rs` that I couldn't read, but it passed the tests. ### Discovering `html5ever` (fast, correct, Rust) While looking for alternatives, I found [`html5ever`](https://github.com/servo/html5ever), [Servo](https://servo.org/)'s parsing engine. It is very correct and written from scratch in Rust to be fast. ### Asking: why build this at all? At that point I had the uncomfortable thought: why would the world need a slower version of `html5ever` in partial Python? What is the meaning of it all?! I almost just deleted the whole project. ### Pivoting to porting `html5ever` logic to Python Instead of quitting, I considered writing a Python interface against `html5ever`, but decided I didn't like the hassle of a library requiring installing binary files. So I went pure Python again, but with a faster approach: what if I port the `html5ever` logic to Python? Shouldn't that be faster than the existing Python libraries? I decided to throw all previous work away. ### Restarting from scratch (again) So I started over from <1% test coverage and iterated with the same set of tests all the way up to 100%. This time I asked it to cross reference the Rust codebase in the beginning. It was tedious work, doing the same thing over again. ### Still slower than `html5lib` Unfortunately, I ran the benchmark on the new codebase and found that it was *still* slower than `html5lib`. ### Profiling, real-world benchmarks, and micro-optimizations So I switched to performance work: I wrote some new tools for the agents to use, a simple profiler and a scraper that built a dataset of 100k popular webpages for real-world benchmarking. I managed to get the speed down below the target with Python micro-optimizations, but only when using the just-released Gemini 3 Pro (which is incredible) to run the benchmark and profiler iteratively. No other model made any progress on the benchmarks. ```python def _append_text_chunk(self, chunk, *, ends_with_cr=False): if not chunk: self.ignore_lf = ends_with_cr return if self.ignore_lf: if chunk[0] == "\n": chunk = chunk[1:] # ... ``` ### Deleting untested code (coverage as a scalpel) Later, on a whim I ran [`coverage`](https://coverage.readthedocs.io/) on the codebase and found that large parts of the code were "untested". But this was backwards, because I already knew that the tests were covering everything important. So lines with no test coverage could be removed! I told the agent to start removing code to reach 100% test coverage, which was an interesting reversal of roles. These removals actually sped up the code as much as the microoptimizations. ```python # Before: 786 lines of treebuilder code # After: 453 lines of treebuilder code # Result: Faster and cleaner ``` ### Fuzzing to find crashes and harden the parser After removing code, I got worried that I had removed too much and missed corner cases. So I asked the agent to write a [html5 fuzzer](https://github.com/EmilStenstrom/justhtml/blob/master/benchmarks/fuzz.py) that tried really hard to generate HTML that broke the parser. ```python def generate_fuzzed_html(): """Generate a complete fuzzed HTML document.""" parts = [] if random.random() < 0.5: parts.append(fuzz_doctype()) # Generate random mix of elements num_elements = random.randint(1, 20) # ... ``` It did break the parser, and for each breaking case I asked it to fix it, and write a new test for the test suite. Passed 3 million generated webpages without any crashes, and hardened the codebase again. ### Comparing against other parsers (how rare 100% is) To sanity-check where 100% landed, I ran the `html5lib` tests against the other parsers. I found that **no other parser passes 90% coverage**, and that `lxml`, one of the most popular Python parsers, is at **1%**. The reference implementation, html5lib itself, is at 88%. Maybe this is a hard problem after all? ### Shipping it as a library (CI, releases, selector API) Finally, to make this a good library I asked the agent to set up CI, releases via GitHub, a [query API](https://github.com/EmilStenstrom/justhtml/blob/master/src/justhtml/selector.py), write READMEs, and so on. ```python from justhtml import JustHTML, query doc = JustHTML("
Hello