Our team is looking into Robotic Process Automation (RPA) and we need to extract invoice data reliably. My concern is the "probabilistic" nature of LLMs. How does the Instructor library ensure that we don't end up with hallucinated fields that still pass JSON validation? Is there a way to add custom validation logic?
3 answers
For RPA, accuracy is everything. While Instructor ensures the "shape" of the data is correct, you can combat hallucinations by using Pydantic's @field_validator. For example, if you are extracting an invoice date, you can write a custom function to check if that date is in a valid range or matches a specific format. If the LLM makes something up that doesn't pass your logic, the Instructor library will treat it as a failure and ask the model to fix it. This adds a crucial layer of business logic validation that simple JSON parsing lacks.
Can these validators also check against external databases to verify if a client ID actually exists?
We use it for medical record indexing. The ability to use Enums ensures the model only picks from our pre-approved category list.
Enums are a lifesaver in RPA. It prevents the model from being "creative" with category names, which is a huge win for automation.
Yes, Scott! Since Pydantic validators are just Python code, you can certainly perform database lookups or API calls within them. However, keep an eye on performance. If you're doing complex lookups during the validation phase, it might slow down the retry loop. It's often better to do a quick format check during extraction and then do the deep database validation in a separate step after the LLM has given you a structured output.