Storing the errors alongside other transformed inputs is a really good idea. But manually patching up and/or deleting individual records is a really bad idea! How is that going to help your data integrity?

Surely you want to fix your importer and re-run it. If there was a smart tool that was able to avoid duplicating work and only re-import the affected records, that'd be great, but that doesn't seem to be what's shown here.

Oh wow ! I was just trying to articulate exactly this kind of approach and looking for it. I would love to see a coherent approach to data errors and this seems like a step in the right direction.

One question - the blog post covers basically debugging the ingestion of data part. My quite usual issue with older data is that at some point, you discover an issue with it (say it's slightly false, but not too much) - so you want to somehow let users know, or allow to select only the data without the issue (but still let them know how much of it they miss) - is this framework helpful in this situation ?

It took me a minute to realize that this isn’t about the Zed text editor.

This article would improve by linking to what Zed is supposed to be: https://zed.brimdata.io/docs

"Wouldn’t it be great if you could see errors in place instead of mysterious NULLs?"

It's not a good ad when the error message is inadequate even in the supplied example and you need to hack around it.

Thought it was about Zed Shaw for a moment and got ready for something colorful

This is really cool. I’d imagine this helps with datasets skewed by errors and actually allows incident losses to be quantized in ways that are usually pretty cumbersome. I think SREs would love this

Interesting. Seems like a hassle to store efficiently.

Unfortunate product name