Safety-Critical Verification and Validation

Critical systems are distinct in that they require a significant amount of Verification and Validation (V&V) activities in order to comply with industry certification requirements. In fact, V&V can easily account for the biggest chunk of effort in the development of safety-critical systems. In industries with the strictest regulatory requirements, V&V alone can amount to 70% of the total development effort. Regulations also often mandate that system development and V&V teams operate completely independently from one another, to prevent conflicts of interest arising. This requires organisations undertaking system development projects to partner with skilled, independent V&V experts, fully immersed in the technical and industrial contexts in which they operate.

CRITICAL Software’s comprehensive V&V services combine mature Software Verification Facilities (SVF) with a world-class team of experts to help organisations meet safety requirements in a timely, cost-effective way. We offer our clients a unique 4-stage approach to accommodate their specific V&V requirements:

  • V&V Setup & Go: designed to support clients in setting up an effective V&V environment and to bring them up to speed with the use of key tools and resources.
  • V&V Analyser: designed to help clients understand the gaps that exist in their projects according to the relevant industry standards in their area.
  • V&V Accelerator: designed to provide a quick response team to help clients overcome project peaks and other challenging or unplanned V&V scenarios.
  • V&V Plus: designed as a bespoke, comprehensive service to provide clients with the full-spectrum of V&V activities.

We have an international pedigree providing V&V services for some of the world’s leading institutions in demanding industries like aerospace, space, defence and transport. Our first ever client in this area was NASA, and safety-critical V&V and testing services have remained at the heart of our business since the very beginning.

Our extensive knowledge of embedded systems and the world’s most important safety standards mean we are able to assist organisations across the entire development life cycle. From airborne systems to spacecraft to railway signalling systems, our V&V experts are supported by our own multi-purpose Software Validation Facilities which add flexibility to our services by supporting automated hardware, model and software-in-the-loop test environments according to the most stringent industry standards, including DO-178B/C Level A. Through our world-class facilities, we are able to re-execute entire test campaigns at any point during development stages, delivering effective non-regression verification whenever the system subject to testing is modified.

The safety standards we support include ARP-4754/4761, DO-178B/C, DO-254, EN 50126/8/9, IEC 61508, ISO 26262, IEC 62304, ECSS Series and many more besides. Our services span the full stack of embedded systems, software, FPGA/ASIC/CPLD and integrated equipment:

  • Planning of V&V activities to ensure conformity with safety standards and certification regulations
  • Independent V&V of system requirements and design
  • Formal verification of source code, unit and integration testing
  • V&V of custom logic designs (ASIC, FPGA and CPLD)
  • Hardware and software integration testing, as well as testing of integrated embedded systems
  • Recording of V&V evidence required for certification
  • Automated testing with software-in-the-loop (SIL), model-in-the-loop (MIL) and hardware-in-the-loop (HIL)
  • Development and tailoring of automated test facilities for embedded software and hardware systems
  • Test facilities and services for equipment acceptance and homologation testing

Our V&V services for critical systems are backed by a delivery unit that is rated at Capability Maturity Model Integration (CMMI)® Level 5, the highest obtainable level, and a roster of quality certifications that include ISO 9001:2008, NATO AQAP 2210 and 2110 and AS 9100.

Safety-Critical V&V for Aerospace

Commercial air travel is one of the safest means of transportation. Aircraft safety figures have been systematically improving since 1945, to the point where an individual’s odds of being a casualty in a lightning strike are higher than their chances of being a casualty in an aircraft accident. Such an impressive safety record for a system as complex as an aircraft could not be achieved without an equally impressive array of Verification & Validation (V&V) activities aimed at ensuring that everything performs as it should.

V&V activities in aerospace projects are governed by a strict and comprehensive ecosystem of industry standards that include DO-178B/C, DO-254, ARP-4754A, ARP-4761 and many others. The challenge for specific projects and the industry at large is to continue improving the safety record of increasingly complex systems and to do so in a business efficient way. To address this challenge, the industry must rely on organisations and individuals that embody a culture of safety and that are skilled at performing V&V activities thoroughly and efficiently. This becomes even more of a challenge as normative evolutions, like the introduction of DO-178C, place even further emphasis on V&V activities. Similar evolutions are to be expected of the industry as it adapts to DO-254A too, whenever it arises, and of other standards like ARP-4754B and ARP-4761A.

At CRITICAL Software, safety is our culture and our raison d'être. Aerospace V&V activities at CRITICAL Software are performed by a dedicated engineering team hardened through many DAL-A and DAL-B projects. The team is skilled at meeting tight time frames while thoroughly executing all V&V activities and delivering the respective formal evidence.

CRITICAL Software’s V&V services for aerospace include:

  • Software Requirements & Design Verification: including high-level requirements (HLR), low-level requirements (LLR), software design and algorithm accuracy. This covers, but is not limited to, DO-178B/C’s objectives in tables A-3 and A-4.
  • Source Code Verification: including exhaustive code scrutiny to assert correctness and traceability against LLRs, correctness and completeness of integration and of the 'parameter data item' file. This covers, but is not limited to, DO-178B/C’s objectives in table A-5.
  • Software Integration (SI) and HW/SW Integration (HIS) Testing: including module testing, software integration (SI) testing and HW/SW integration (HSI) testing. CRITICAL Software favours SI and HSI over module testing and makes use of its own Multi-Purpose System Validation Facility (MPSVF) to automate tests with hardware-in-the-loop. Whenever the MPSVF is used, CRITICAL Software also provides the respective tool qualification data for either DO-178B or DO-330. This covers, but is not limited to, DO-178B/C’s objectives in tables A6 and A-7. The MPSVF can be used to fulfil objectives beyond DO-178B/C’s scope.
  • ASIC/FPGA Verification & Testing: including verification of requirements, design documentation, HDL source code (Verilog, VHDL), and testing of the design on simulator. This covers DO-254/ED-80’s verification objectives as applicable to complex CPLD, FPGA and ASIC, including the provisions of EASA certification memorandum EASA-CM-SWCEH-001.

Safety-Critical V&V for Automotive

The automobile is so prevalent in our daily lives that we often fail to acknowledge the massive array of technologies packed into the modern car. Advances in electronic control units (ECUs) were key to many of the efficiency and safety improvements we now enjoy when using the automobile. These advances turned the car from a mechanical machine to an immensely complex cyber-physical system, changing the driver's role in subtle yet impressive ways. In the future, the driver may be removed from his or her position behind the wheel altogether.

The landscape we live in is marked by more complex ECUs controlled by increasingly complex software, a wide range of variants and shrinking development cycles. While the cycles for new models are still relatively large, cycles for ECU/SW variants, which are deployed across many models, are at the epicentre of time-to-market pressures. Within this context, the impact of failure on manufacturer reputations has dramatically grown and safety concerns in the industry are on the rise, as evidenced by the introduction of ISO 26262.

The automotive industry now has to deal with ever more complex systems, shorter time frames and less tolerance to failure from markets and end users. This is a formidable challenge for Verification & Validation (V&V). One can hardly rely on in-vehicle testing as the main feedback line on ECU safety, performance and functional correctness. The industry needs innovative and efficient approaches to V&V built upon lessons learnt and best practices. This is where CRITICAL Software stands: a company that started with a fault injection tool for space applications and has since performed V&V for ECUs and software deployed in aircraft, spacecraft, rail rolling stock, medical devices, automotive systems and more.

CRITICAL Software’s V&V services for the automotive industry include:

  • Testing of BSP/RTOS: testing of low-level software including board support packages (BSP), real-time operating systems (RTOS) and, in particular, multi-core mixed criticality systems (i.e. testing the inner foundations of the modern ECU).
  • Testing of ECUs and critical software applications: testing of hardware and software safety functions in environments that span across software-in-the-loop (SIL), model-in-the-loop (MIL) and hardware-in-the-loop (HIL). This includes a strict fulfilment of industry standards (such as ISO 26262) and a focus on efficient testing through effective automation and design of tests for reuse across variants.

Safety-Critical V&V for Railway

Rail is one of the oldest transportation systems in the world, and the one that made the early expansion of the industrial revolution possible. To this day, it is still one of the most efficient ways of transporting goods and people. In an age marked by the rapid introduction of new technologies and the unrelenting growth of interconnectivity, many fail to see rail as a realm of innovation. However, the fact is that, because of its efficiency, rail transport has entered a new revival age that is no longer confined to the few countries around the world that have led the development of high-speed networks.

The rail landscape for the 21st century provides significant opportunities for those able to address its challenges. Accurately forecasting how future railway systems will look may seem like a hard task. Some things are almost certain, however: increases in passenger comfort and safety and a more competitive global market. Generic products and generic and specific applications will be more complex, and project execution times will be shorter. The key challenge is how to perform the V&V of ever more complex systems within narrower time frames, while delivering increased levels of safety assurance. This is the challenge CRITICAL Software is devoted to.

Our vast experience across a wide range of safety-critical domains enables us to deliver an effective and efficient V&V solution to increasingly complex and competitive projects. CRITICAL Software offers a comprehensive set of V&V services for the rail industry that include:

  • TCMS Software Testing: including component and integration testing in full conformance with the provisions of CENELEC EN 50128.
  • HMI Testing: testing of human-machine interfaces such as the software applications running on the consoles operated by the train driver.
  • System Testing: definition of test cases for different vehicle functions and execution of test procedures in either a test rack or train simulator. Case studies of functions tested by CRITICAL Software include main line voltage, auxiliary power, doors control, TCMS, etc.
  • In-Vehicle Testing: definition of test cases for tests on the vehicle and execution of the respective test procedures. This is done in close co-operation with the client's technical and management personnel.

Safety-Critical V&V for Space

If it were to be suggested to an audience that spacecraft are launched before being tested, it might well trigger a strong sense of surprise and disbelief. Such reactions would be well-justified because the suggestion is not entirely true. However, it is also not completely in opposition to the facts. As surprising as it may seem, and contrary to common practice in other industries, spacecraft are not fully tested before being launched, and neither are test flights, except for rockets and other very specific missions.

But, there are good reasons for this lack of testing. The first is that it is simply not possible to fully test a spacecraft before launch, because we cannot fully replicate the conditions of space on the ground. The second reason is that launching a spacecraft just for the sake of testing it makes for a test case that is far too expensive to be economically viable.

The fact that spacecraft cannot be fully tested on the ground makes V&V activities even more challenging. Having participated in over fifteen space missions to date, CRITICAL Software is well aware of this challenge, and well versed on how to combine software-in-the-loop (SIL), model-in-the-loop (MIL) and hardware-in-the-loop (HIL) to achieve maximum V&V coverage on the ground, in the most efficient way.

Though testing in space is, in general, not viable, lessons can and must be learned from previous missions. Which errors occurred in the past? How were they detected and overcome? Which architectures were most and least effective? Which specification flaws or deficiencies were present? Experience is paramount in addressing all these questions and more.

CRITICAL Software’s V&V services for space include:

  • Independent software Verification & Validation (ISVV): including, but not limited to, ISVV according to the mission-specific tailoring of ESA's ISVV Guide, a document that CRITICAL Software originally contributed to.
  • On-board software validation (TS/RB Validation): where the scope may include the validation of the entire on-board software (OBSW), specific functional blocks only, or otherwise start from an incomplete validation state.
  • AIT support: support to functional verification and/or AIT including: functional testing of central software in FV/AIT test bench, functional testing of equipment (e.g. Star Tracker), and support to proto-flight model (PFM) tests. Some of CRITICAL Software’s case studies include the CryoSat-2 mission (AIT and PFM support), ExoMars TGO (FV support including interfacing between FV, AIT and OBSW teams) and, to a lesser extent, S2 (interface between OBSW and AIT).
  • On-board software V&V support: execution of specific V&V activities across the OBSW life cycle, either extending or complementing the costumer project team already in place. This service can be instantiated to respond to a project peak, contingency situation or simple outsourcing of a range of V&V activities.

Want to know more about our work in safety-critical V&V?

Click the button below to download our latest white paper on safety-critical validation.

Download white paper

Talk to us today!

Click the button below and one of our experts will contact you shortly.

Contact us