curl --location --request POST 'https://api-prod.extend.app/evaluation_set_items' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_TOKEN>' \
--data '{
  "evaluationSetId": "eval_set_123",
  "fileId": "file_456",
  "expectedOutput": {
    "value": {
      "invoice_number": "36995",
      "total_amount": 15735.1,
      "line_items": [
        {
          "description": "Premium Widget",
          "quantity": 5,
          "unit_price": {
            "amount": 200.00,
            "iso_4217_currency_code": "USD"
          },
          "total": {
            "amount": 1000.00,
            "iso_4217_currency_code": "USD"
          }
        },
        {
          "description": "Basic Widget",
          "quantity": 10,
          "unit_price": {
            "amount": 25.05,
            "iso_4217_currency_code": "USD"
          },
          "total": {
            "amount": 250.50,
            "iso_4217_currency_code": "USD"
          }
        }
      ]
    },
    "metadata": {},
  }
}'

{
  "success": true,
  "evaluationSetItem": {
    "object": "evaluation_set_item",
    "id": "eval_item_1234",
    "evaluationSetId": "eval_set_123",
    "fileId": "file_456",
    "expectedOutput": {
      // Output object - see “Processor output types” for details
    },
    "createdAt": "2024-01-01T00:00:00Z",
    "updatedAt": "2024-01-01T00:00:00Z"
  }
}

Evaluation set items are the individual files and expected outputs that are used to evaluate the performance of a given processor in Extend. This endpoint will create a new evaluation set item in Extend, which will be used during an evaluation run.

​
Best Practices for Outputs in Evaluation Sets

  1. Configure First, Output Later
  • Always create and finalize your processor configuration before creating evaluation sets
  • Field IDs in outputs must match those defined in your processor configuration
  1. Type Consistency
  • Ensure output types exactly match your processor configuration
  • For example, if a field is configured as “currency”, don’t submit a simple number value
  1. Field IDs
  • Use the exact field IDs from your processor configuration
  • Create your own semantic IDs instead in the configs for each field/type instead of using the generated ones
  1. Value
  • Remember that all results are inside the value key of a result object, except the values within nested structures.

​
Body

​
evaluationSetId
string
required

The ID of the evaluation set to add the item to.

​
fileId
string
required

The ID of the file to add to the evaluation set.

​
expectedOutput
object
required

The expected output of the processor when run against the file. This should be a JSON object conforming to the output type schema of the processor.

curl --location --request POST 'https://api-prod.extend.app/evaluation_set_items' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <API_TOKEN>' \
--data '{
  "evaluationSetId": "eval_set_123",
  "fileId": "file_456",
  "expectedOutput": {
    "value": {
      "invoice_number": "36995",
      "total_amount": 15735.1,
      "line_items": [
        {
          "description": "Premium Widget",
          "quantity": 5,
          "unit_price": {
            "amount": 200.00,
            "iso_4217_currency_code": "USD"
          },
          "total": {
            "amount": 1000.00,
            "iso_4217_currency_code": "USD"
          }
        },
        {
          "description": "Basic Widget",
          "quantity": 10,
          "unit_price": {
            "amount": 25.05,
            "iso_4217_currency_code": "USD"
          },
          "total": {
            "amount": 250.50,
            "iso_4217_currency_code": "USD"
          }
        }
      ]
    },
    "metadata": {},
  }
}'

​
Response

​
success
boolean

A true or false value for whether the evaluation set item was created successfully or not.

​
evaluationSetItem
EvaluationSetItem

An EvaluationSetItem object representing the newly created evaluation set item. See the EvaluationSetItem object for more details.

{
  "success": true,
  "evaluationSetItem": {
    "object": "evaluation_set_item",
    "id": "eval_item_1234",
    "evaluationSetId": "eval_set_123",
    "fileId": "file_456",
    "expectedOutput": {
      // Output object - see “Processor output types” for details
    },
    "createdAt": "2024-01-01T00:00:00Z",
    "updatedAt": "2024-01-01T00:00:00Z"
  }
}