Description
We collected mission requirements in natural language from available requirements produced from Spectra and LTLMoP. We checked how the pattern catalog may have supported developers in the definition of the mission requirements. In the case of Spectra, we use the Spectra files to extract mission requirements for robotic systems. In total, 11 robotic applications were considered. Note that mission requirements are realistic since they were finally executed with real robots. We automatically extracted 428 mission requirements from the Spectra files. In the case of LTLMoP, $16$ requirements were extracted.
Each mission requirement was independently analyzed by two of the authors. The authors checked whether it is possible to express the mission requirement using the mission specification patterns. If one the authors stated that the requirement is not expressible the requirement is marked as not expressible (NE). Then, the mission specification patterns used to express the mission requirement are considered. If the authors used the same mission specification patterns to express the mission requirement, a consensus is reached.
Results
The results show that most of the mission requirements (370 over 436) were expressible using the pattern catalog, which is a reasonable coverage for pattern catalog usage. The 66 mission requirements that are not covered suggested the introduction of new patterns. It also shows that the pattern catalog is effective in real case scenarios. In 102 cases the mission requirements were ambiguous, meaning that different interpretations can be given to the proposed mission requirement. In these cases, alternative combinations of patterns have been proposed by the authors to express the mission requirement. Each of these alternatives represents a possible way of expressing it in a non-ambiguous manner. In 156 cases, while the authors judged that the requirement was not ambiguous, different pattern combinations were proposed. The combinations of patterns encode possible ways of expressing the mission requirement in a non-ambiguous manner.
Reproduction Kit
To Extract requirements, run the following scripts. Results will be in missions.txt.
ExtractingRequirements/SYNTECH15/requirementScript.sh ExtractingRequirements/SYNTECH17/requirementScript.sh
For the manual tagging:
RequirementsAndManualTagging/ICSEMissionRequirements.xlsx
To process consensus run ProcessingRequirements/requirementScript.sh
To get pattern prevalence run
ProcessingRequirements/statistics.sh
Specifications & Data
| Archive | File Contents # |
|---|---|
| ProcessingRequirements.zip | 20 |
| ExtractingRequirements.zip | 6 |
| RequirementsAndManualTagging.zip | 1 |