API-SemEnrich: Experimental Data for Automated Extraction of Business Rules from REST API Documentation Using Large Language Models

Abstract

Experimental data supporting the paper "Automated Extraction of Business Rules from REST API Documentation Using Large Language Models." The study is a descriptive characterization of business-rule extraction by open-weight large language models from REST API documentation: three models (DeepSeek-V4-Flash, Llama-4-Scout, GPT-OSS-120B) applied to three documentation corpora (Stripe, GitHub, Twilio), a 3x3 matrix of nine runs yielding 6,763 extracted business rules. All models were served through OpenRouter with a pinned provider and fallbacks disabled, decoding fixed at temperature 0. This deposit contains: - runs/ : the nine experimental cells. Each holds the extracted rules (rules.jsonl), per-stage statistics and attrition counters (stats.json), token and cost records (cost.json), a full provenance manifest recording the configuration, prompt hash, corpus hashes and provider pin (run-manifest.json), the configuration as executed, the prompt templates, and a human-readable summary.- corpora/ : the three chunked documentation corpora used as extraction input, as markdown pages plus the chunk and index records the models consumed.- schemas/ : JSON Schemas for the x-business-rules OpenAPI extension and for the MCP and Arazzo export formats.- configs/ : the nine per-cell experiment configurations, nine single-pass baseline configurations, and the model pricing table.- validation/ : evidence record for the Qwen-family zero-yield finding.- DATA-MANIFEST.md : source URLs, retrieval dates, SHA-256 hashes and the chunking configuration for every corpus. Each extracted rule carries a verbatim evidence quote and a source_chunk_id that joins to the corpus chunk it was drawn from, so every rule can be traced to the exact input text the model received. README.md documents the record format field by field and states which counts reproduce which published figure. This is a descriptive dataset. It characterizes extraction behaviour (yield, category distribution, reclassification, source faithfulness, cost) and does not measure extraction accuracy. The rules are model outputs, not a validated gold standard, and no human-annotated ground truth is included or claimed. The extraction pipeline's source code is not part of this deposit. It remains under development for follow-on work and is available from the corresponding author on reasonable request for verifying the reported results.