Supplied binary probability calibration
Compute exact binary Brier score and uniform-bin calibration diagnostics from supplied probabilities and nullable Boolean outcomes. Reports label coverage and per-row errors; no learned calibration, confidence intervals, forecast validity or population-quality claim.
data-assurance · Operation ID: prediction-calibration-check
Choose this operation when
- Evaluate supplied agent confidence probabilities against binary outcomes
- Inspect Brier error and reliability bins without dropping unknown labels silently
Outside this profile
- Extracting facts or judging truth, semantic correctness, evidence authenticity, causal effects, or production readiness
- Fetching data, executing models or user code, changing datasets, authorizing actions, or contacting live services
- Inferring population quality, statistical significance, uncertainty intervals, or generalization from a supplied sample
Exact release references
Static JSON contract · Markdown reference · Fixed example response
Complete input schema
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"predictions": {
"minItems": 1,
"maxItems": 1000,
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"maxLength": 100
},
"probability": {
"type": "string",
"maxLength": 44,
"pattern": "^-?(?:0|[1-9]\\d{0,23})(?:\\.\\d{1,18})?$",
"description": "Exact decimal string: up to 24 integer and 18 fractional digits; no exponent, plus sign, whitespace, or leading zeros."
},
"outcome": {
"type": [
"boolean",
"null"
]
}
},
"required": [
"id",
"probability",
"outcome"
],
"additionalProperties": false
}
},
"binCount": {
"default": 10,
"type": "integer",
"minimum": 1,
"maximum": 20
}
},
"required": [
"predictions"
],
"additionalProperties": false
}
Complete output-envelope schema
{
"type": "object",
"required": [
"operation",
"version",
"result",
"provenance"
],
"properties": {
"operation": {
"const": "prediction-calibration-check",
"type": "string"
},
"version": {
"const": "0.29.0",
"type": "string"
},
"result": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"profile": {
"type": "string",
"const": "binary-probability-calibration-v1"
},
"advisoryOnly": {
"type": "boolean",
"const": true
},
"externalRequests": {
"type": "number",
"const": 0
},
"semanticValidityVerified": {
"type": "boolean",
"const": false
},
"predictionCount": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"labeledCount": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"unknownOutcomeCount": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"complete": {
"type": "boolean"
},
"brierScore": {
"anyOf": [
{
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"expectedCalibrationError": {
"anyOf": [
{
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"maximumCalibrationGap": {
"anyOf": [
{
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"bins": {
"maxItems": 20,
"type": "array",
"items": {
"type": "object",
"properties": {
"index": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"lowerInclusive": {
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
"upper": {
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
"upperInclusive": {
"type": "boolean"
},
"predictionCount": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"labeledCount": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"positiveCount": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"meanProbability": {
"anyOf": [
{
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"observedRate": {
"anyOf": [
{
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
{
"type": "null"
}
]
},
"calibrationGap": {
"anyOf": [
{
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
{
"type": "null"
}
]
}
},
"required": [
"index",
"lowerInclusive",
"upper",
"upperInclusive",
"predictionCount",
"labeledCount",
"positiveCount",
"meanProbability",
"observedRate",
"calibrationGap"
],
"additionalProperties": false
}
},
"predictions": {
"maxItems": 1000,
"type": "array",
"items": {
"type": "object",
"properties": {
"id": {
"type": "string",
"minLength": 1,
"maxLength": 100
},
"binIndex": {
"type": "integer",
"minimum": 0,
"maximum": 9007199254740991
},
"squaredError": {
"anyOf": [
{
"type": "object",
"properties": {
"numerator": {
"type": "string",
"maxLength": 2049,
"pattern": "^-?(?:0|[1-9]\\d*)$"
},
"denominator": {
"type": "string",
"maxLength": 2048,
"pattern": "^[1-9]\\d*$"
}
},
"required": [
"numerator",
"denominator"
],
"additionalProperties": false
},
{
"type": "null"
}
]
}
},
"required": [
"id",
"binIndex",
"squaredError"
],
"additionalProperties": false
}
}
},
"required": [
"profile",
"advisoryOnly",
"externalRequests",
"semanticValidityVerified",
"predictionCount",
"labeledCount",
"unknownOutcomeCount",
"complete",
"brierScore",
"expectedCalibrationError",
"maximumCalibrationGap",
"bins",
"predictions"
],
"additionalProperties": false
},
"provenance": {
"type": "object",
"required": [
"inputSha256",
"outputSha256",
"deterministic",
"externalRequests"
],
"properties": {
"inputSha256": {
"type": "string",
"pattern": "^[a-f0-9]{64}$"
},
"outputSha256": {
"type": "string",
"pattern": "^[a-f0-9]{64}$"
},
"deterministic": {
"const": true
},
"externalRequests": {
"const": 0
}
}
}
},
"additionalProperties": false
}
Fixed example
One accepted fixed example, not a custom-input trial. No operation runs when this static page is requested.
Example input
{
"binCount": 2,
"predictions": [
{
"id": "case-1",
"probability": "0.8",
"outcome": true
},
{
"id": "case-2",
"probability": "0.8",
"outcome": false
},
{
"id": "case-3",
"probability": "0.2",
"outcome": null
}
]
}
Example response
{
"operation": "prediction-calibration-check",
"version": "0.29.0",
"result": {
"profile": "binary-probability-calibration-v1",
"advisoryOnly": true,
"externalRequests": 0,
"semanticValidityVerified": false,
"predictionCount": 3,
"labeledCount": 2,
"unknownOutcomeCount": 1,
"complete": false,
"brierScore": {
"numerator": "17",
"denominator": "50"
},
"expectedCalibrationError": {
"numerator": "3",
"denominator": "10"
},
"maximumCalibrationGap": {
"numerator": "3",
"denominator": "10"
},
"bins": [
{
"index": 0,
"lowerInclusive": {
"numerator": "0",
"denominator": "1"
},
"upper": {
"numerator": "1",
"denominator": "2"
},
"upperInclusive": false,
"predictionCount": 1,
"labeledCount": 0,
"positiveCount": 0,
"meanProbability": null,
"observedRate": null,
"calibrationGap": null
},
{
"index": 1,
"lowerInclusive": {
"numerator": "1",
"denominator": "2"
},
"upper": {
"numerator": "1",
"denominator": "1"
},
"upperInclusive": true,
"predictionCount": 2,
"labeledCount": 2,
"positiveCount": 1,
"meanProbability": {
"numerator": "4",
"denominator": "5"
},
"observedRate": {
"numerator": "1",
"denominator": "2"
},
"calibrationGap": {
"numerator": "3",
"denominator": "10"
}
}
],
"predictions": [
{
"id": "case-1",
"binIndex": 1,
"squaredError": {
"numerator": "1",
"denominator": "25"
}
},
{
"id": "case-2",
"binIndex": 1,
"squaredError": {
"numerator": "16",
"denominator": "25"
}
},
{
"id": "case-3",
"binIndex": 0,
"squaredError": null
}
]
},
"provenance": {
"inputSha256": "487d8a324ebc4d910aa27e48d5abb89bb0c16022113f707bc6dc6c321cde07b4",
"outputSha256": "ec152f997950d908cbe1a00982c312bab17bb2f591623e4872555689c6013503",
"deterministic": true,
"externalRequests": 0
}
}
Bounds and precision
JavaScript IEEE-754 numbers; use strings for large integer IDs/exact decimals where the schema accepts strings. No lossless numeric parsing.
{
"global": {
"requestBytes": 131072,
"responseBytes": 524288,
"jsonDepth": 32,
"jsonNodes": 20000,
"requestsPerMinute": 60,
"paidAttemptsPerMinute": 20,
"idempotencyHours": 24
},
"operation": {
"inputBytes": 100000,
"outputBytes": 400000,
"jsonNodes": 12000,
"depth": 20,
"findings": 200,
"decimalIntegerDigits": 24,
"decimalFractionDigits": 18,
"rationalDigits": 2048,
"predictions": 1000,
"binCount": "1–20"
}
}
Complete schemas, descriptions and cross-field validation may impose additional limits.
Proposed price and protocol definitions
{
"unit": "one successful operation call",
"proposedNominalUsd": "0.005",
"sixDecimalTokenBaseUnits": "5000",
"subscription": false,
"includesPayerWalletOrNetworkFees": false,
"liveQuoteVerified": false,
"condition": "Actual SDK challenge is authoritative only within the caller's explicit authorization; configured six-decimal token peg is an operator assertion, not a conversion guarantee."
}
Protocol definitions: x402, mpp. MPP uses Tempo charge. Paid MCP execution is unsupported. All runtime readiness is not evaluated in this build.
API path templates, not endpoints on this documentation host
{
"x402": "/v1/x402/prediction-calibration-check",
"mpp": "/v1/mpp/prediction-calibration-check"
}
Required headers
{
"Content-Type": "application/json",
"Idempotency-Key": "random 16–128 character operation identifier"
}
Actual SDK challenge amount, asset, network, recipient and wallet costs must pass independent authorization. Preserve identical key, body, protocol and credential on retries; on PAYMENT_UNCERTAIN stop and reconcile.
Execution profile and provider conditions
{
"deterministic": true,
"externalRequests": 0,
"maxExternalRequests": 0,
"resultSnapshotPersisted": false,
"fixedExampleIsIllustrativeSnapshot": false,
"requiresPayment": true,
"supportsMcpExecution": false
}
Deterministic supplied-input operation with no external requests or stored request/result bodies. Payment infrastructure retains payment metadata and hashes.
Failure handling
- HTTP 400: Malformed JSON, missing/invalid idempotency key, or payment identifier mismatch Correct the request before payment
- HTTP 402: Payment challenge or rejected payment Use official protocol SDK; inspect payment outcome before another payment
- HTTP 409: Idempotency conflict, duplicate proof, or PAYMENT_UNCERTAIN Keep original key, body, and proof; reconcile uncertainty with operator; never blindly repay
- HTTP 413: Input or generated output too large Reduce input; no payment attempted for validation failure
- HTTP 415: Unsupported media type or compression Send uncompressed application/json
- HTTP 422: Schema or service-specific semantic validation failure Correct input using returned error code; no payment attempted
- HTTP 429: Request/payment-attempt rate exceeded Wait for rate limit window; preserve existing payment identity
- HTTP 503: Payment configuration/provider/state unavailable, or live DNS preparation failed before settlement Check readiness; DNS preparation failures may retry the identical key/body/credential only; uncertainty requires reconciliation
Declared requirements
Before any paid call, refresh the live operation contract and POST the complete bounded budgeted plan to the separate API's /preflight. Unknown requirements block selection; compatible preflight is not permission to spend.