Question: Is checking whether data already exists the same problem as recovering data that never got in?

Task 4 was supposed to be a small safety check. It ended up being a script that touched 402 files.

I thought I was writing a guard.

The database kept telling me something else.

There was nothing to verify because there was almost nothing there.

The friction

By the time I got to Task 4, the registration API from Task 2 was already working. New images were getting a proper image_asset_id, a content_hash, all of it. But every time I opened an old project — one from before any of this existed — I had no idea what state it was in. Was it already registered somewhere? Would re-registering it create duplicates? I didn't want to find out by breaking something.

What I thought I needed

So Task 4 was scoped as a guard: before letting an old project register its images, check whether meta.projectId already exists in the DB. If it does, treat it as already handled and skip it. A one-line safety net, nothing more — the kind of task you write in a sentence and expect to close in an afternoon.

Why that didn't hold

Why didn't the check alone work? Because "does this project exist in the DB" was answering the wrong question. Almost none of the old projects existed in the DB. They weren't duplicates waiting to be skipped — they simply predated the registration system. The check wasn't guarding against a mistake. It was staring at an empty table.

Why did that matter for something meant to be just a safety check? Because if the guard did exactly what it was designed to do — block anything not already verified — it would have permanently locked out every project that came before Task 2. The safety net and the actual goal were pointing in opposite directions. A check built to prevent bad writes was, by construction, going to prevent all the real writes too.

Why not just let each old project register itself the normal way, one at a time? Because the entire reason the asset DB exists is so that search and recommendation (which hadn't been built yet, but were already planned) could see everything. If old projects only entered the system when someone happened to reopen them, the DB would always be missing an unknown, growing slice of history — invisible until someone searched for an image that should have been there and wasn't.

The task wasn't "verify before writing." It was "get the history in, because right now it isn't."

The decision

The answer was to pull the work out of the live registration path entirely and write it as a standalone script instead — something that scans all 402 existing JSON files once, registers what it finds, and updates the files in place. In practice, it was a migration.

The alternative was to fold that logic into the registration API itself: have it detect "old, unregistered project" as just another case to handle on the fly. I didn't go that way. The API's job is to answer one question correctly every time it's called — does this content already exist, yes or no — and that has to stay fast and simple, because it runs on every save from here on out. Historical backfill is a different shape of problem: it runs once, it can take its time, and it can afford to be messy in ways a live endpoint can't. Keeping them separate meant the registration path stayed exactly as simple as it was designed to be, and the one-time cleanup could be as slow and thorough as it needed to be without that cost ever showing up again.

The script ran against all 402 JSON files: 2,492 scenes scanned, 2,083 images registered, 1,202 new assets created, 278 JSON files updated in place.

Takeaway

I thought Task 4 was a guard rail — something that stops a mistake before it happens. It turned out to be a migration — something that recovers history that was never captured in the first place. Those aren't the same task, even though they can look identical from the outside: both start with "check if this already exists."

The migration proved the data could enter the system. It didn't prove it entered correctly.

Most of the work isn't building the solution. It's discovering what the problem actually is.

Wrong assumption: Task 4 was a check to stop duplicate registration. Discovery: There was nothing to duplicate — the data had never entered the system at all. Next step: Registering history is one thing. Proving every asset belongs to the right project is another.