The Problem

I’ve been writing book reviews on this blog for a while, and one thing has always been a drag: creating a new post. For each review, I had to go to honto.jp, copy the title and author, create a new markdown file, fill in the Hugo front matter, and add all the boilerplate. It’s not hard, but it’s a solid 5-10 minutes of tedious clicking and typing. That friction adds up, and it often became a barrier to actually sitting down to write.

I’d been thinking about automating it for a while, and with local LLMs getting better, I figured now was the time. This is about that project—building it with Gemini, Claude, and a locally-running gemma3:12b model.

The Plan

My goal was simple: take a honto.jp URL and have a script spit out a perfectly formatted, pre-filled Hugo post, ready for me to write the actual review.

The project ended up as a set of Python scripts:

• book_review_helper.py: Extracts metadata from honto.jp
• create_review.py: Orchestrates the whole process from URL to final .md file
• validate_random_sample.py: Testing framework to measure accuracy

The implementation was a constant back-and-forth with AI assistants. I’d ask Gemini to sketch out the overall structure, then hand the rough code to Claude for refactoring and error handling. It worked surprisingly well.

How It Works

Metadata Extraction

When I feed it a URL, book_review_helper.py kicks off a sequence:

1. HTML Caching: It checks a local html_cache/ directory first. If it’s seen the URL before, it uses the cached version. This was a time-saver during development and prevented rate-limiting from honto.jp.
2. Content Extraction: Uses BeautifulSoup to parse the HTML and extract about 3000 characters of relevant text from the main product block.
3. LLM Inference: The script feeds that text into a locally running gemma3:12b model via Ollama. I crafted specific prompts (one for novels, one for manga) that instruct the model to return a JSON object with the book’s title, author, illustrator, and tags.
4. Post-Processing: The raw JSON gets cleaned up. For instance, it splits a single author string like “Author Name, Illustrator Name” into separate author and illustrator fields.

Building the Post

The create_review.py script then takes over:

1. It calls the helper to get the metadata.
2. It calls the LLM again with a different prompt to romanize the Japanese title into a URL-friendly slug.
3. It runs hugo new to create the post file from the correct archetype (manga.md or novel.md).
4. Finally, it injects all the extracted metadata into the new file.

And that’s it - a perfectly formatted draft ready to write in.

Model Selection

Before settling on gemma3:12b, I spent some time testing different models. The goal was to balance speed, accuracy, and cost for structured data extraction.

I wanted the script to be fast and free to run, which pointed me toward a local solution with Ollama. I tested a few different models. Larger models were slightly more capable but slower. Smaller models were fast but often failed to follow the strict JSON output format I needed.

gemma3:12b hit the sweet spot. It was fast enough (about 1-2 seconds per extraction on my machine), consistently returned valid JSON, and was accurate at identifying the correct author and illustrator fields once the prompt was tuned.

Testing It

I couldn’t just trust a few successful runs. I needed data. So I built validate_random_sample.py. This script scans my nearly 300 existing book review posts, extracts the “ground truth” metadata from them, and runs my new script against their honto URLs to compare the results. It calculates an accuracy score and tracks which posts it has tested in a .validation_state.json file.

This validation framework turned out to be the most important part. It turned my vague sense of “it seems to work” into hard numbers.

Getting to 84% Accuracy

The first run of the validation script gave me 77.4%. Not bad, but the failures were almost all the same: it couldn’t find the illustrator for light novels.

I dug into the HTML of the failed pages and realized I was wrong about how the data was structured. I assumed there would always be a clear “イラスト” (illustrator) label. But honto.jp often just lists the author and illustrator under a single “著者” (author) tag, separated by a comma. My script was seeing this, getting confused, and incorrectly classifying the book as a manga, so it never even looked for an illustrator.

My first fix was to improve the prompt for the gemma3:12b model, explicitly telling it how to handle that comma-separated case. I re-ran the test. Accuracy jumped to 82.2%. Better!

But the root cause was still the faulty auto-detection in my Python code. The prompt was a patch, not a fix. So I went back to the code and made the detection logic smarter. If it sees that 著者 Name1, Name2 pattern, it now classifies the book as a novel.

After that change, accuracy hit 83.7%.

Where It Stands

The script is at 83.7% accuracy across about a quarter of my posts. The path to 90% is clear: tackle the remaining edge cases, like manga with three or more creators, and improve the validation logic to handle things like full-width vs. half-width character differences.

This was an interesting experiment in working with AI tools. I was the architect and debugger. Gemini helped with prompts and high-level code structure. Claude did the refactoring, turning messy first drafts into clean Python. It’s a workflow I’ll use again.

Update: Getting to 95%

After writing this initial post, I continued iterating with Claude. The breakthrough came from fixing how the script extracts content from HTML pages. By prioritizing the product detail section over promotional banners, the LLM stopped hallucinating author names and accuracy jumped to 95.1%.

I also added automatic slug romanization—the LLM now converts Japanese titles like 痴漢されそうになっているＳ級美少女... into clean, 20-character URLs like chikan-s-rank-bishou. One less manual step.

The system now reduces what was 5-10 minutes of tedious work down to about 30 seconds, and it’s accurate enough that I trust it for initial drafts.

Original post written by Gemini, updates by me with Claude’s help