Deduplicate lines

Use to remove repeated records from newline-delimited text while keeping the first original spelling and order. Comparison optionally trims whitespace and uses locale-independent lowercase. Output separators become LF; no fuzzy matching or Unicode normalization. Empty text has zero lines; a trailing newline represents a final blank line.

text · Operation ID: lines-dedupe

Choose this operation when

Keep the first spelling and order with optional trim/lowercase comparison.

Outside this profile

Fuzzy entity resolution or Unicode normalization.

Exact release references

Static JSON contract · Markdown reference · Fixed example response

Complete input schema

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "type": "object",
  "properties": {
    "text": {
      "type": "string",
      "maxLength": 100000
    },
    "trimForComparison": {
      "default": false,
      "type": "boolean"
    },
    "caseSensitive": {
      "default": true,
      "type": "boolean"
    },
    "dropEmpty": {
      "default": false,
      "type": "boolean"
    }
  },
  "required": [
    "text"
  ],
  "additionalProperties": false
}

Complete output-envelope schema

{
  "type": "object",
  "required": [
    "operation",
    "version",
    "result",
    "provenance"
  ],
  "properties": {
    "operation": {
      "const": "lines-dedupe",
      "type": "string"
    },
    "version": {
      "const": "0.29.0",
      "type": "string"
    },
    "result": {
      "$schema": "https://json-schema.org/draft/2020-12/schema",
      "type": "object",
      "properties": {
        "text": {
          "type": "string",
          "maxLength": 100000
        },
        "inputLines": {
          "type": "integer",
          "minimum": 0,
          "maximum": 100001
        },
        "outputLines": {
          "type": "integer",
          "minimum": 0,
          "maximum": 100001
        },
        "duplicateLinesRemoved": {
          "type": "integer",
          "minimum": 0,
          "maximum": 100001
        },
        "emptyLinesRemoved": {
          "type": "integer",
          "minimum": 0,
          "maximum": 100001
        },
        "keptOriginalLineNumbers": {
          "maxItems": 100001,
          "type": "array",
          "items": {
            "type": "integer",
            "minimum": 1,
            "maximum": 100001
          }
        }
      },
      "required": [
        "text",
        "inputLines",
        "outputLines",
        "duplicateLinesRemoved",
        "emptyLinesRemoved",
        "keptOriginalLineNumbers"
      ],
      "additionalProperties": false
    },
    "provenance": {
      "type": "object",
      "required": [
        "inputSha256",
        "outputSha256",
        "deterministic",
        "externalRequests"
      ],
      "properties": {
        "inputSha256": {
          "type": "string",
          "pattern": "^[a-f0-9]{64}$"
        },
        "outputSha256": {
          "type": "string",
          "pattern": "^[a-f0-9]{64}$"
        },
        "deterministic": {
          "const": true
        },
        "externalRequests": {
          "const": 0
        }
      }
    }
  },
  "additionalProperties": false
}

Fixed example

One accepted fixed example, not a custom-input trial. No operation runs when this static page is requested.

Example input

{
  "text": "Alpha\nBeta\n alpha \nBeta",
  "trimForComparison": true,
  "caseSensitive": false
}

Example response

{
  "operation": "lines-dedupe",
  "version": "0.29.0",
  "result": {
    "text": "Alpha\nBeta",
    "inputLines": 4,
    "outputLines": 2,
    "duplicateLinesRemoved": 2,
    "emptyLinesRemoved": 0,
    "keptOriginalLineNumbers": [
      1,
      2
    ]
  },
  "provenance": {
    "inputSha256": "c2483912e3f47691860b328a56b737710dd3ddfe2215cd13d62d858978378a9b",
    "outputSha256": "0b13ea8b088f1220910f61475e1909bf90ca5a39f789eab1977ba255f1716b14",
    "deterministic": true,
    "externalRequests": 0
  }
}

Bounds and precision

JavaScript IEEE-754 numbers; use strings for large integer IDs/exact decimals where the schema accepts strings. No lossless numeric parsing.

{
  "global": {
    "requestBytes": 131072,
    "responseBytes": 524288,
    "jsonDepth": 32,
    "jsonNodes": 20000,
    "requestsPerMinute": 60,
    "paidAttemptsPerMinute": 20,
    "idempotencyHours": 24
  },
  "operation": {
    "requestBytes": 131072,
    "responseBytes": 524288,
    "jsonDepth": 32
  }
}

Complete schemas, descriptions and cross-field validation may impose additional limits.

Proposed price and protocol definitions

{
  "unit": "one successful operation call",
  "proposedNominalUsd": "0.003",
  "sixDecimalTokenBaseUnits": "3000",
  "subscription": false,
  "includesPayerWalletOrNetworkFees": false,
  "liveQuoteVerified": false,
  "condition": "Actual SDK challenge is authoritative only within the caller's explicit authorization; configured six-decimal token peg is an operator assertion, not a conversion guarantee."
}

Protocol definitions: x402, mpp. MPP uses Tempo charge. Paid MCP execution is unsupported. All runtime readiness is not evaluated in this build.

API path templates, not endpoints on this documentation host

{
  "x402": "/v1/x402/lines-dedupe",
  "mpp": "/v1/mpp/lines-dedupe"
}

Required headers

{
  "Content-Type": "application/json",
  "Idempotency-Key": "random 16–128 character operation identifier"
}

Actual SDK challenge amount, asset, network, recipient and wallet costs must pass independent authorization. Preserve identical key, body, protocol and credential on retries; on PAYMENT_UNCERTAIN stop and reconcile.

Execution profile and provider conditions

{
  "deterministic": true,
  "externalRequests": 0,
  "maxExternalRequests": 0,
  "resultSnapshotPersisted": false,
  "fixedExampleIsIllustrativeSnapshot": false,
  "requiresPayment": true,
  "supportsMcpExecution": false
}

Deterministic supplied-input operation with no external requests or stored request/result bodies. Payment infrastructure retains payment metadata and hashes.

Failure handling

HTTP 400: Malformed JSON, missing/invalid idempotency key, or payment identifier mismatch Correct the request before payment
HTTP 402: Payment challenge or rejected payment Use official protocol SDK; inspect payment outcome before another payment
HTTP 409: Idempotency conflict, duplicate proof, or PAYMENT_UNCERTAIN Keep original key, body, and proof; reconcile uncertainty with operator; never blindly repay
HTTP 413: Input or generated output too large Reduce input; no payment attempted for validation failure
HTTP 415: Unsupported media type or compression Send uncompressed application/json
HTTP 422: Schema or service-specific semantic validation failure Correct input using returned error code; no payment attempted
HTTP 429: Request/payment-attempt rate exceeded Wait for rate limit window; preserve existing payment identity
HTTP 503: Payment configuration/provider/state unavailable, or live DNS preparation failed before settlement Check readiness; DNS preparation failures may retry the identical key/body/credential only; uncertainty requires reconciliation

Declared requirements

Before any paid call, refresh the live operation contract and POST the complete bounded budgeted plan to the separate API's /preflight. Unknown requirements block selection; compatible preflight is not permission to spend.