Publications

Although model-driven software engineering (MDSE) has proven effective in managing complex systems, its industrial adoption remains limited by the substantial maintenance overhead required for models and the specialised skills demanded of developers. Meanwhile, advances in artificial intelligence (AI), particularly generative and agentic AI, have shown great promise in automating code-related tasks such as comprehension, generation, and defect detection. These capabilities are largely powered by ’big code’: vast repositories of open-source software that now form the basis of data-driven, empirical SE and automated quality assurance. This paper aims to synthesise these two domains by exploring the integration of AI into model-driven practices. It provides a comprehensive overview of the current state of AI-augmented software engineering and introduces a novel taxonomy ’ ai4se ’ to classify and connect diverse AI applications within the field. On this basis, the paper proposes a vision for ’big models’ in software engineering (SE), an approach designed to leverage the structural advantages of MDSE alongside the scalability of AI. Finally, the paper discusses the pair modelling paradigm as a collaborative framework for the MDSE industry, designed to enhance software quality through human–AI partnership.

Software profoundly influences how we live and work, often reflecting human behavior and societal values. Consequently, software-intensive systems and software engineering technologies must be developed and deployed responsibly to avoid harming users, society, or the environment. This raises questions about what constitutes the responsible development and use of technology, including which human values and which social and environmental norms are embedded in software engineering practices and products. With the increased incorporation of AI in software-intensive systems, software engineers are confronted with even more ethical and moral choices to incorporate into these processes and products. To address these responsibility issues, the workshop hosted a half-day event featuring themed presentation tracks, interactive discussions, and a stimulating concluding panel. The workshop comprised research papers (visionary, exploratory, empirical, and solution-oriented) and industrial papers (lessons learned and challenges) published in the ACM workshop proceedings.

Agentic artificial intelligence (AI) marks a new phase of digitalization: digitalization is beginning to act back upon its own technical production base. Whereas earlier phases aimed at digitizing analog information, automating processes, and building digital value networks, AI is increasingly taking over tasks in the development of digital systems themselves. In software engineering (SE) in particular, agentic AI systems can already plan, execute, check, and iteratively refine development tasks across multiple steps. Digitalization thus becomes recursive: digital systems no longer merely support the digitalization of other sectors, but are increasingly developed and evolved by digital systems themselves. The technical production base of digitalization thereby becomes the object of its own transformation. This changes not only the division of labor between humans and machines in SE, but also the role of SE as a whole. This chapter analyzes this socio-technical change and argues that SE is evolving from a primarily supporting engineering discipline into a central infrastructure of societal value creation as well as of technological and digital sovereignty.

Artificial intelligence (AI) represents one of the most powerful technologies currently available to humanity, offering immense potential for transformative change. It has the potential to assist us in addressing our most pressing climate change concerns. It can also enhance our ability to promote and lead the transition to sustainability. However, it is also important to acknowledge that AI is a technology that requires significant resources. It is estimated that by 2025–2026, the daily training costs for the large AI models will exceed the global computing capacity. This chapter presents the latest developments in AI for sustainability transformation and sustainable AI, with a particular focus on reducing resource consumption. It introduces the concept of sustainable intelligence and discusses a research agenda featuring a multidisciplinary, transformative approach to the design and development of AI technologies and AI-based systems, with the objective of ensuring their development in an environmentally friendly and socially responsible manner, and with the aim of investigating their impact on the natural, technical and societal environment.

The landscape of software testing is undergoing a significant transformation due to the advent of Artificial Intelligence (AI), particularly along Generative AI (GenAI) and agentic AI. This extended abstract explores the integration of AI into software testing, highlighting its capabilities in various test activities, outlining key challenges such as misinformation and biases, and presenting the ai4st taxonomy as a framework to understand and classify research in this rapidly evolving field. It also discusses the strategic adoption of AI in testing organisations and the future outlook for AI-augmented software testing.

Artificial Intelligence (AI) is one of the most powerful tools available to humanity. While some debate the potential dangers of AI in general and consider a moratorium on AI development and applications necessary, others propose concepts such as AI for Sustainability and Sustainable AI to harness the potential of AI to address humanity's most pressing challenges, including climate change to combat environmental degradation - in short, Sustainable Intelligence. In order to ground the still heated debate, this paper delves deeper into the state of the art of Sustainable Intelligence, in particular how to qualitatively and quantitatively explore the environmental impact of AI-based systems and how to minimise it. It also discusses selected research topics that need to be further explored and concludes with a Rising Star system for Sustainable Intelligence to share knowledge on environmental and societal goals for AI-based systems and to report and optimise their qualitative indicators and quantitative metrics in an open and standardised way.

The wealth of artificial intelligence methods enables digital solutions, without which the transition to sustainability, including the transition to renewable energy and the circular economy, would be impossible. It would be unethical not to harness these enormous opportunities, not to liberalise access to data, computing, and AI resources worldwide, and not to educate and train people around the world to make the most of AI. My personal AI mission: On a large scale, my dream is the liberalization of AI. Basic AI should not be a masterpiece of political, economic, or military power. A key area for the beneficial application of this AI vision is to use this powerful tool to address the most critical threat to humanity, which is sustainability in all its forms. On a smaller scale, I would like to contribute to the improvement of software engineering by applying AI methods and tools.

In industry, software testing is the primary method to verify and validate the functionality, performance, security, usability, and so on, of software-based systems. Test automation has gained increasing atten- tion in industry over the last decade, following decades of intense research into test automation and model-based testing. However, designing, devel- oping, maintaining and evolving test automation is a considerable effort. Meanwhile, AI’s breakthroughs in many engineering fields are opening up new perspectives for software testing, for both manual and automated testing. This paper reviews recent research on AI augmentation in soft- ware test automation, from no automation to full automation. It also discusses new forms of testing made possible by AI. Based on this, the newly developed taxonomy, ai4st, is presented and used to classify recent research and identify open research questions.

This paper reviews the development of Model-Driven Software Engineering (MDSE) from its origins to the present day. The paper discusses the components of models, modelling activities, and the challenges and open issues in MDSE. It reviews the adoption of modelling languages and their use in MDSE processes. Selected tool sets and frameworks are presented. The paper concludes with a review of the still unresolved relationship and implications of Artificial Intelligence (AI) for software engineering, including MDSE, and the coming developments in the Internet of Collaboration for distributed MDSE. Throughout the paper, Tiziana Margaria’s research and dissemination contributions in these areas are highlighted.

Aspirations toward technological sovereignty increasingly pervade the political debate. Yet, an ambiguous definition leaves the exact goal of those aspirations and the policies to fulfil them unclear. This opens the door for vested interests who benefit from misinterpreting the goal, e.g., as a strive for autarky, nationalism, and the rollback of globalization. To close this gap, we show how certain key technologies challenge state sovereignty as conventionally understood. By interpreting technological sovereignty in this light, we develop a competence-based definition, which puts innovation policy at the core of fulfilling sovereignty aspirations. Moreover, we establish the important role of international cooperation and trade to enhance technological sovereignty understood as ability. Hence, autarky would be detrimental rather than helpful to technological sovereignty. Two case studies illustrate how innovation policy helps to achieve technological sovereignty.

In den letzten Jahren gab es verschiedenste Ansätze zur Weiterentwicklung von Städten und Gemeinden mittels digitaler Angebote und digitalisierter Infrastrukturen. Egal, ob es dabei um neue Beteiligungs-, ÖPNV- oder Bildungsangebote geht, spielen öffentlich zur Verfügung gestellte Daten eine zentrale Rolle. So betraf eine der ersten Aktivitäten weltweit die Öffnung großer Datenmengen nach Open-Data-Prinzipien von verschiedenen öffentlichen Verwaltungen und Versorgungsunternehmen innerhalb einer Stadt, um ein funktionierendes Ökosystem für städtische Dienste und Anwendungen rund um diese Daten zu schaffen. Dabei wurde schnell deutlich, dass offene Daten der öffentlichen Hand zwar ein wichtiges, aber nicht das einzige Datenangebot in einem solchen Ökosystem sein sollten, sondern ein breites Spektrum an Daten verschiedenster Anbieter und somit unter verschiedenen Nutzungskonditionen benötigt werden, um attraktive datengetriebene Angebote zu ermöglichen. Dabei werden Teile dieser Datenangebote als Gemeingüter benötigt, so dass jede und jeder an der Nutzung und Weiterverarbeitung partizipieren kann. Zudem wurde klar, dass die bereitgestellten Daten und Metadaten auf ihre Qualität zu überprüfen und ein entsprechendes Qualitätsniveau zu gewährleisten ist. Zudem entstand das Konzept einer offenen urbanen Plattform als Gesamtansatz für eine öffentlich-rechtliche Informations- und Kommunikationsinfrastruktur – als Architektur für eine digitale Infrastruktur für Informationsangebote und -dienste im öffentlichen Raum und in öffentlicher Verantwortung. Dieser Artikel bietet einen Überblick zu Daten und Metadaten im Sinne von Gemeingütern, zu ihrer Verankerung in einer öffentlich-rechtlichen Informations- und Kommunikationsinfrastruktur und zu verfügbaren Software-Komponenten und Informationen, also zu Ansatzpunkten für Interessierte.

Information and Communication Technology (ICT) is at the heart of the smart city approach, which constitutes the next level of cities’ and communities’ development across the globe. Thereby, ICT serves as the gluing component enabling different domains to interact with each other and facilitating the management and processing of vast amounts of data and information towards intelligently steering the cities’ infrastructure and processes, engaging the citizens and facilitating new services and applications in various aspects of urban life—e.g., supply chains, mobility, transportation, energy, citizens’ participation, public safety, interactions between citizens and the public administration, water management, parking and many other cases and domains. Hence, given the fundamental role of ICT in cities in the near future, it is of paramount importance to lay the ground for a sustainable and reliable ICT infrastructure, which can enable a city/community to respond in a resilient way to upcoming challenges, whilst increasing the quality of life for its citizens. A structured way of providing and maintaining an open and resilient ICT backbone for a city/community is constituted by the concept of an Open Urban Platform. Therefore, the current article presents the activities and developments necessary to achieve a resilient, standardized smart city, based on Open Urban Platforms (OUP) and the way these serve as a blueprint for each city/community towards the establishment of a sustainable and resilient ICT backbone.

Aspirations towards technological sovereignty increasingly pervade the political debate. Yet, an ambiguous definition leaves the exact goal of those aspirations and the policies to fulfill them unclear. This leaves room for partly particularly negative interpretations, such as equating the concept with a strive for autarky, nationalism, and the roll-back of globalization. We develop a competence-based definition of technological sovereignty, which puts innovation policy at the core of fulfilling sovereignty aspirations. Moreover, we show how our definition realigns technological sovereignty with international cooperation and trade. Two case studies illustrate how innovation policy might be used to achieve technological sovereignty.

This article provides a taxonomy for risk-based testing that serves as a tool to define, tailor, or assess risk-based testing approaches in general and to instantiate risk-based testing approaches for the current testing standards ISO/IEC/IEEE 29119, ETSI EG and OWASP Security Testing Guide in particular. We demonstrate the usefulness of the taxonomy by applying it to the aforementioned standards as well as to the risk-based testing approaches SmartTesting, RACOMAT, PRISMA and risk-based test case prioritization using fuzzy expert systems. In this setting, the taxonomy is used to systematically identify deviations between the standards’ requirements and the individual testing approaches so that we are able to position and compare the testing approaches and discuss their potential for practical application.

Digital innovations are changing our economies and societies. Shaping this transformation in line with core societal values and in a continual exchange with society is a key political challenge. The authors argue that technological sovereignty lies at the heart of this challenge. Rather than the autarchy from foreign technologies, this comprises the ability to define and establish core requirements on and properties of technologies, which safeguard the self-determination of citizens in the digital age. The authors show how an integrated innovation policy, which rests upon European collaboration and combines the promotion of technology development, technology transfer and individuals’ competencies, can achieve this goal.

Edge Computing is a new distributed Cloud Computing paradigm in which computing and storage capabilities are pushed to the topological edge of a network. However, various standards and implementations are promoted by different initiatives. Lead by a reference architecture model for Edge Computing, current initiatives are analyzed by explorative content analysis. Providing two main contributions to the field, we present, first, how current initiatives are characterized, and second, a roadmap for sustainable Edge Computing relating three dimensions of sustainable development to four cross-concerns of Edge Computing. Findings show that most initiatives are internationally organized software development projects; important branches are currently telecom and industrial sectors; most addressed is the network virtualization layer. The roadmap reveals numerous chances and risks of Edge Computing related to sustainable development; such as the use of renewable energies, biases, new business models, increase and decrease of energy consumption, responsiveness, monitoring and traceability.

Firewalls are a critical part in any security framework. Most firewalls consist of a large amount of sequential rules that are unstructured and confusing. Unfortunately, because a lot of rules configuration work is done manually by the network administrators, misconfigurations are very common and can affect the reliability of the firewall. Identifying such anomalies is a challenging task. In this paper, we propose a tree based simulation and verification model to verify if the implemented firewall of a system is in compliance with the belonging firewall requirements. The proposed methodology was developed in relation with the H2020 FORTIKA project and was evaluated in the scope of case studies with industrial partners. The case studies in question related to large scale telecom infrastructures involving critical scenarios in the scope of Smart Cities in general and SME cyber-security protection. Thereby, the executed case studies demonstrate how our approach can lead to improved structuring of firewalls and belonging rules, to the comfortable visualization of firewall structures and decision patterns, and finally to the verification of system and context requirements imposed by the firewall operation environment.

During the past years, various activities and concepts have shaped and prepared the path for the development of urban environments toward smart cities across the world. One of the initial activities was relating to the opening of vast amounts of data from various public administrations and utility companies within a city in order to create a viable eco-system of urban services and applications. Thereby, the harvested metadata needed to be verified in terms of correctness and a corresponding level of quality had to be assured. In addition, the concept of an Open Urban Platform emerged as an overall solution for smart cities Information Communication Technology (ICT) in the sense that an abstract reference model was established and standardized, providing an overall picture of the ICT structures within a city. Within this article, we use the Open Urban Platform concept as the basics to describe and map our activities within the Open Data domain, focusing mainly on the Open Data prototype for German Open Governmental Data—namely GovData.DE. Thereby, we describe our metadata harvesting and metadata quality assurance approach and discuss on lessons learned, which flow into the definition of metadata quality metrics and have the potential to lead to a corresponding standard within the Deutsches Institut für Normung e.V. (DIN) German national standardization.

Only if there is a fundamental change in the way we manage land can we reach the targets of climate-change mitigation, avert the dramatic loss of biodiversity and make the global food system sustainable. The WBGU proposes five multiple-benefit strategies illustrating ways of overcoming competition between rival claims to the use of land. These should be promoted by five governance strategies, especially by setting suitable framework conditions, reorienting EU policy and establishing alliances of like-minded states.

The institutional framework for global sustainable development in the Digital Age needs a normative reference point in the form of an international charter for a sustainable Digital Age. The WBGU submits here a draft for such a charter. It ties in with the 2030 Agenda and the Declaration of Human Rights and, at the same time, goes beyond them. The charter is intended to serve as a system of principles, objectives and standards for the international community and to link digital change with the necessary global sustainability perspective. It formulates objectives and principles for the protection of human dignity, natural life-support systems, inclusion in and access to digital and digitalized infrastructures and technologies, as well as individual and collective freedom of development in the Digital Age. On this basis, the charter sets out concrete guidelines for action to be drawn up by the international community with a view to the challenges of the Digital Age. The charter contains three core elements: First, digitalization should be designed in line with the 2030 Agenda, and digital technology should be used to achieve the SDGs. Second, beyond the 2030 Agenda, systemic risks should be avoided, in particular by protecting civil and human rights, promoting the common good and ensuring decision-making sovereignty. Third, societies must prepare themselves procedurally for future challenges by agreeing, among other things, on ethical guidelines and ensuring future-oriented research and education.

The new European Commission president has announced a Green Deal for Europe. This can only succeed if opportunities and risks of digitalization for a fundamental transformation of the economy and society are taken into account. In this paper, the WBGU develops cornerstones of a European way to a common digital future. The EU’s cross-cutting sustainability policy needs an implementation strategy for the SDGs and an 8th Environmental Action Programme that exploits digital technologies for its objectives and minimizes the risks associated with them. The EU’s digital policy should implement ecological and social objectives in addition to economic ones and therefore demand, for example, the provision of data by companies and sustainability-oriented AI. Public-service digital infrastructures and services should also be guaranteed. Research and innovation policy should strategically promote digital technologies to achieve sustainability goals. In this way, the EU can become an international role model in interlinking digitalization and sustainability.

Software rules them all! In every industry now, software plays a dominant role in technical and business innovations, in improving functional safety, and also for increasing convenience. Nevertheless, software is not always designed, (re)developed, and/or secured with the necessary professionalism, and there are unnecessary interruptions in the development, maintenance, and operating chains that adversely affect reliable, secure, powerful, and trustworthy systems. Current surveys such as the annual World Quality Report put it bluntly, directly correlated with the now well-known failures of large-scale, important and/or safetycritical infrastructures caused by software. It is thus high time that software development be left to the experts and that space be created for the use of current methods and technologies. The present article sheds light on current and future software engineering approaches that can also and especially be found in the Fraunhofer portfolio.

European cities and communities (and beyond) require a structured overview and a set of tools as to achieve a sustainable transformation towards smarter cities/municipalities, thereby leveraging on the enormous potential of the emerging data driven economy. This paper presents the results of a recent study that was conducted with a number of German municipalities/cities. Based on the obtained and briefly presented recommendations emerging from the study, the authors propose the concept of an Urban Data Space (UDS), which facilitates an eco-system for data exchange and added value creation thereby utilizing the various types of data within a smart city/municipality. Looking at an Urban Data Space from within a German context and considering the current situation and developments in German municipalities, this paper proposes a reasonable classification of urban data that allows the relation of various data types to legal aspects, and to conduct solid considerations regarding technical implementation designs and decisions. Furthermore, the Urban Data Space is described/analyzed in detail, and relevant stakeholders are identified, as well as corresponding technical artifacts are introduced. The authors propose to setup Urban Data Spaces based on emerging standards from the area of ICT reference architectures for Smart Cities, such as DIN SPEC 91357 “Open Urban Platform” and EIP SCC. In the course of this, the paper walks the reader through the construction of a UDS based on the above-mentioned architectures and outlines all the goals, recommendations and potentials, which an Urban Data Space can reveal to a municipality/city. Finally, we aim at deriving the proposed concepts in a way that they have the potential to be part of the required set of tools towards the sustainable transformation of German and European cities in the direction of smarter urban environments, based on utilizing the hidden potential of digitalization and efficient interoperable data exchange.

Die Digitalisierung öffnet die Tür zu einem neuen Zeitalter menschlicher Zivilisation. Große Gestaltungsaufgaben entstehen, um den digitalen Wandel mit der notwendigen Transformation zur Nachhaltigkeit, also der Entwicklung von Wohlstand, Sicherheit und Demokratie für bald 10 Milliarden Menschen in den Grenzen des Erdsystems, zusammenzuführen. Die Transformation zur Nachhaltigkeit muss bis etwa Mitte dieses Jahrhunderts gelingen, um Kipppunkte im Erdsystem – wie das Abschmelzen des Grönlandeisschildes, den Kollaps des Monsunsystems oder des Amazonasregenwaldgebietes – zu vermeiden. Es geht darum, Wohlstandssteigerung und die sozio-ökonomische Entwicklung in den kommenden drei bis vier Dekaden von Treibhausgasemissionen zu entkoppeln, Ressourcen in wirtschaftliche Kreisläufe zu führen und den Druck auf die Ökosysteme radikal zu reduzieren. Mit der Agenda 2030 und ihren umfassenden Nachhaltigkeitszielen geht es zudem um die Abschaffung der extremen Armut und die Reduzierung von Ungleichheiten, die soziale Kohäsion und Stabilität vieler Gesellschaften gefährden. Kann die Digitalisierung helfen, diese Menschheitsaufgaben zu bewältigen? Dazu müssen digitale Innovationen mit der Transformation zur Nachhaltigkeit zusammengeführt werden. Darauf sind Politik, Wirtschaft und Gesellschaft bisher nicht eingestellt.

The United Nations’ High-Level Political Forum for Sustainable Development is the UN’s key platform for coordinating global sustainability policy and an important addressee for issues relating to the potential benefits and risks of digital change for achieving the UN sustainability goals. Digitalization fundamentally changes the range of options available for future civilizational development. At the same time, the implementation of the 2030 Agenda with its 17 Sustainable Development Goals requires a fundamental transformation of the economy and society. These two challenges can only be met by dedicated global thinking and action. So what could be more appropriate than to use the momentum of digitalization to advance the Transformation towards Sustainability and launch decisive stimuli for achieving national and global sustainability targets?

In recent years the emergence of the Ethereum Blockchain has introduced a new alternative perspective on how web applications can be build. More precisely, the Ethereum Blockchain allows the development of applications, where programming code can be executed in a decentralized manner with no restrictions imposed by a central authority. However, as it is the case with many emerging technologies, there is a fair amount of trade-offs that have to be considered when this technology is used as a platform for implementing decentralized applications. In this work we present two architectural designs for building decentralized applications (DApps) based on the Ethereum Blockchain technology. Within this context, we discuss the inherent strengths and weaknesses of each of the architectural designs as well as the set of challenges that we faced during the development process.

‘Digitalization’ is often described as a huge upheaval facing our societies to which we must adapt. The WBGU opposes this interpretation, saying that digitalization must be shaped in such a way that it can serve as a lever and support for the Great Transformation towards Sustainability, and can be synchronized with it. The WBGU understands digitalization broadly as the development and application of digital and digitalized technologies that augment and dovetail with all other technologies and methods. It has a profound effect on all economic, social and societal systems and is developing an ever greater transformative force. This in turn is increasingly having a fundamental impact on people, societies and the planet itself and must therefore be managed accordingly. Just as in 1987 the Brundtland Report entitled ‘Our Common Future’ outlined the concept of Sustainable Development, the WBGU’s report entitled ‘Towards Our Common Digital Future’ sketches the concept of a digitalized sustainability society.

Software trustworthiness today is more about acceptance than technical quality; software and its features must be comprehensible and explainable. Since software becomes more and more a public good, software quality becomes a critical concern for human society. And insofar artificial intelligence (AI) has become part of our daily lives—naturally we use language assistants or automatic translation programs—software quality is evolving and has to take into account usability, transparency as well as safety and security. Indeed, a majority worldwide rejects currently the use of AI in schools, in court or in the army because it is afraid of data misuse or heteronomy. Insofar, software and its applications can succeed only if people trust them. The initiatives towards “responsible software engineering” address these concerns. This publication is about raising awareness for responsible software engineering.

This extended abstract was submitted as answer to the call for proposals for the DFG programme NFDI (national research-data infrastructure). The initiative is spear-headed by de-RSE - Gesellschaft für Wissenschaftssoftware, but many other institutes and individuals contributed and are involved as well.

The quality assurance of large scale integrative systems often requires complex testbed environments and simulations that allow to test the overall functionality and enables various experiments towards systematically verifying the realization of the identified user and system requirements. Thereby, an integration setup and resulting activities lead to another level of quality assurance, whereby the integrator deals with the quality examination of the single components and their integrative interplay according to a set of overall system and user requirements. In such context, it is often the case that the testing activity is conducted by various partners (e.g. single companies and legal entities) with complementing know how required for specific sub-tasks - e.g. PKI, chip cards, special network protocols, firewall, security architectures, and penetration testing. This leads to the emergence of a large number of proprietary testbeds focusing on specific aspects resulting in the lack of a unified testbed configuration, versioning and technological foundations (e.g. operating system, network stack implementations, hypervisor technology ...). In this paper, we present our experiences drafted from a large scale industrial project with 600-700 requirements relating to a critical eHealth infrastructure within a telecom provider context. Thereby, various sub-contractors had to be unified in their approach to testbed management in order to achieve reproducible and traceable (with respect to system requirements) test results based on a test architecture accommodating various quality assurance activities (unit testing, development tests, component testing, integration testing, security testing ...). We gradually analyze the project situation with respect to testbed management and argue on the need for unified testbed management across multiple teams and stakeholders in a large scale telecom integration setup. Subsequently, we propose possible solutions and conduct a series of experiments highlighting the advantages of the proposed approach and belonging solution.

Digitalization fundamentally changes the possible pathways for future development. At the same time, implementing the 2030 Agenda for Sustainable Development requires a fundamental transformation of the economy and society. So what could be more appropriate than to harness the momentum of digitalization to launch decisive stimuli for the Transformation towards Sustainability? What interventions and collaborations does this require? And what are the risks if such a technological revolution is not properly monitored? The world society is only gradually beginning to discuss such questions. Yet it’s high time. This is an invitation.

Im Zuge des digitalen Wandels werden zunehmend Software-basierte Systeme, die auch Methoden der Künstlichen Intelligenz (KI) nutzen, mit menschlichen Aktivitäten abgestimmt. Getrieben von Daten, die von Menschen erzeugt, aggregiert und / oder aufbereitet werden, arbeiten solche, auch kurz genannt KI-basierten Systeme immer enger mit uns Menschen in Echtzeit zusammen. Die Suche nach den dafür benötigten passenden und leistungsfähigen Automatismen mittels KI ist ebenso eine Suche nach den Erwartungen und Anforderungen, die an solche Systeme zu legen sind und wie diese im Entwurf, in der Entwicklung und im Betrieb abgesichert werden können. Vertrauenswürdigkeit als eine der wesentlichen Grundlagen zur Akzeptanz KI-basierter Systeme muss dazu über Korrektheit sowie Zuverlässigkeit, Leistungsfähigkeit sowie Skalierbarkeit, Sicherheit sowie Datenschutz als auch Nachhaltigkeit solcher Systeme erreicht werden. Dieser Artikel bietet dazu eine Reflektion des Standes des Software Engineerings als Disziplin zur Entwicklung vertrauenswürdiger Software und zudem Parallelen als auch neue Herausforderungen bei der Entwicklung KI-basierter Systeme. Der Artikel ist eine Weiterentwicklung von (Schieferdecker, Großmann, & Schneider, 2019; Schieferdecker & Ritter, 2019).

Software rules them all! In jeder Branche spielt Software mittlerweile eine dominante Rolle für Innovationen technischer oder geschäftlicher Art, für die Verbesserung der funktionalen Sicherheit oder aber für die Erhöhung des Komforts. Gleichwohl wird Software nicht immer mit der nötigen Professionalität entworfen, (weiter-)entwickelt und/oder abgesichert - und gibt es unnötige Brüche in den Entwicklungs-, Wartungs- und Betriebsketten, die zuverlässigen, sicheren, performanten und vertrauenswürdigen Systemen zuwiderlaufen. Aktuelle Umfragen wie im jährlichen World Quality Report sprechen hier eine deutliche Sprache, welche direkt mit den bekannt gewordenen von Software verursachten Ausfällen großer, wichtiger und/oder sicherheitskritischer Infrastrukturen korreliert. So ist es höchste Zeit, die Softwareentwicklung den Experten zu überlassen und den Raum für die Nutzung aktueller Methoden und Technologien zuzulassen. Dieser Artikel wirft einen Blick auf aktuelle und zukünftige Software-Engineering- Ansätze, die Sie auch und insbesondere im Fraunhofer-Portfolio finden.

Das digitale Zeitalter öffnet die Tür zu einem neuen Zeitalter menschlicher Zivilisation. Große Gestaltungsaufgaben entstehen, um die digitale Revolution mit der notwendigen Transformation zur Nachhaltigkeit, also der Entwicklung von Wohlstand, Sicherheit und Demokratie für bald zehn Milliarden Menschen in den Grenzen des Erdsystems, zusammenzuführen. Die Transformation zur Nachhaltigkeit muss bis etwa Mitte dieses Jahrhunderts gelingen, um Kipppunkte im Erdsystem - wie das Abschmelzen des Grönlandeisschildes, den Kollaps des Monsunsystems oder des Amazonasregenwaldgebietes - zu vermeiden.

A major technical revolution is underway. How will it change the coexistence of humankind on this planet? What goals will it serve? What opportunities and risks does it involve? Who will gain or lose power in its wake? How can it be used to solve humankind’s great challenges?

With the emergence of Smart Cities, a large number of technologies have been developed and deployed world-wide. However, as Smart City technology is continuously growing, the question emerges of how to reasonably structure the ICT landscape in an urban environment, such that an eco-system of ICT components, involving vendors, network/service providers, small-mid-size enterprises, large-scale industry, and open source initiatives can emerge. Hence, a reference model serving as a blueprint for ICT architectures for future cities is required - similarly as TCP/IP and ISO/OSI facilitated the growth and world-wide development of the Internet and telecommunication services. The current paper proposes a methodology for structuring the ICT landscape in a city according to emerging standards for Smart Cities on European and German level, towards the facilitation of extensible and evolvable eco-system of ICT components and stakeholders for Smart Cities. After introducing the standards and the methodology, two case studies from recent research projects are presented in order to validate our approach.

Limiting global warming to well below 2°C requires the rapid decarbonization of the global economy. If this enterprise fails, we will jeopardize the life-support systems of future generations. The longer the transformation towards climate compatibility is delayed, the more severe the risks and damage will be for a growing number of people. The transformation requirements and the damage caused by climate change have an unequal temporal, geographical and social distribution – as do the respective possibilities for dealing with them. The WBGU therefore proposes a just & in-time transformation that takes into account all people affected, empowers them, holds those responsible for climate change accountable, and creates both global and national prospects for the future. The WBGU proposes that the German Federal Government should promote four exemplary initiatives of a just & in-time climate policy targeting (1) the people affected by the structural change towards climate compatibility (e.g. in coal-mining regions), (2) the legal rights of people harmed by climate change, (3) the dignified migration of people who lose their native countries due to climate change, and (4) the creation of financing instruments for just & in-time transformation processes.

This work proposes a new graph grammar formalism, that introduces non-terminal symbols to triple graph grammars (TGG) and shows how to apply it to solving the model transformation problem. Our proposed formalism seems to suit code generation from models well, outperforms the standard TGG in the grammar size in one evaluated case and is able to express one transformation that we could not express with TGG. We claim, that such advantages make a formal specification written in our formalism easier to validate and less error-prone, what befits safety-critical systems specially well.

Security-related tests should check and validate in engineering that the functionality derived from the security requirements has been correctly implemented. The design of effective tests for this security functionality requires, in addition to test models, a procedure for obtaining these models from previous artefacts of security engineering. This paper describes such an approach and focuses on the use of the artefacts of previous phases of security engineering for security testing.

In order to optimize the usage of testing efforts and to assess risks of software-based systems, risk-based testing uses risk (re-)assessments to steer all phases in a test process. Several risk-based testing approaches have been proposed in academia and/or applied in industry, so that the determination of principal concepts and methods in risk-based testing is needed to enable a comparison of the weaknesses and strengths of different risk-based testing approaches. In this chapter we provide an (updated)taxonomy of risk-based testing aligned with risk considerations in all phases of a test process. It consists of three top-level classes, i.e., contextual setup, risk assessment, and risk-based test strategy. This taxonomy provides a framework to understand, categorize, assess and compare risk-based testing approaches to support their selection and tailoring for specific purposes. Furthermore, we position four recent risk-based testing approaches into the taxonomy in order to demonstrate its application and alignment with available risk-based testing approaches.

Mit urbanen Datenplattformen verbinden viele Kommunen und Städte einerseits ein noch schwer zu fassendes Konzept und andererseits die Möglichkeit für mehr Innovation, Transparenz, Beteiligung als auch zusätzliche Einnahmen für die Haushaltskasse. Dieser Artikel gibt einen Einblick zum Stand urbaner Datenplattformen und diskutiert das Pros und Cons direkter vs. indirekter Einnahmen aus öffentlichen Daten.

Smart Cities will mainly emerge around the opening of large amounts of data, which are currently kept closed by various stakeholders within an urban ecosystem. This data requires to be cataloged and made available to the community, such that applications and services can be developed for citizens, companies and for optimizing processes within a city itself. In that scope, the current work seeks to develop concepts and prototypes, in order to enable and demonstrate, how data cataloging and data storage can be merged towards the provisioning of large amounts of data in urban environments. The developed concepts, prototype, case study and belonging evaluations are based on the integration of common technologies from the domains of Open Data and large scale data processing in data centers, namely CKAN and Hadoop.

The protection and securing of data platforms and related services plays a major role in the development of safety critical infrastructure for Smart Cities. Therefore, this paper specifies and develops an Integerated Component for Cloud Services (ISCS) that enables secure and trusted access to data and related services in the cloud. That means that the ISCS controls and handles access-related aspects such as authentication, authorization and registration. Furthermore, it is deployed and used in a large scale research project, in which it secures cloud services relating to electric mobility. ISCS is realized using OAuth and OpenID, whereas both are implemented by existing open source libraries. OAuth is a standard allowing services and applications to access safety critical resources in the cloud using a trustworthy infrastructure. OpenID is a popular standard originating from the web community, facilitating cross-domain authentication for portal users. The combination of both standards provides rich functionality, covering substantial aspects related to cloud service protection and security.

A large number of activities have been conducted during the past years in the cross-domain area of Smart Cities. The research and development efforts started with various projects which aimed at opening the large amounts of data kept by the public administration – a goal which would ultimately enable various ICT based applications and services in urban environments.

Smart Cities are emerging based on the idea of provisioning and processing large amounts of urban data for various use cases. Thereby, Urban Data Platforms are usually employed to accumulate and expose the large amounts of governmental (i.e. public sector), sensor, static and real-time data in order to enable the community to create valuable applications and services for future Smart Cities. Hitherto, the Open Data initiative was seen as the key driver to providing large amounts of data within a city. Open Data platforms employ so-called data registries in order to keep track of the available datasets at various sources spread throughout the city, with CKAN currently being among the most popular data catalog software worldwide. With the emergence of frameworks for large scale distributed computing and storage, such as Hadoop and the belonging distributed file systems (HDFS), there is an inherent need for bridging the worlds of metadata catalogs and distributed data processing towards the goal of providing sophisticated urban ICT services. The current paper constitutes a first attempt on this new field, by prototyping and evaluating components that enable the collaboration and interplay between CKAN and Hadoop/HDFS. This interplay is realized through extensions to CKAN and its harvesting process and its benefits are demonstrated by belonging case studies.

IT-Plattformen sind die Grundlage datengetriebener, serviceorientierter Geschäftsmodelle. Sie ermöglichen es Unternehmen verschiedener Branchen, flexibel und weitgehend automatisiert in digitalen Ökosystemen zusammenzuarbeiten. Für die industrielle Produktion ist die Harmonisierung von IT-Plattformen von zentraler Bedeutung. Noch aber fehlt im produzierenden Gewerbe ein einheitliches Verständnis von Plattformen und ihrem künftigen Zusammenspiel. Die vorliegende acatech DISKUSSION analysiert das in Wissenschaft und Wirtschaft vorherrschende Verständnis von IT-Plattformen. Auf Basis von Interviews und Workshops mit Fachleuten beleuchtet sie Begrifflichkeiten, Merkmale und Stakeholder sowie den aktuellen Wissensstand zur Plattformökonomie. Weiterhin thematisiert sie Chancen, Risiken und Barrieren bei der industriellen Nutzung von IT-Plattformen sowie damit verbundene Herausforderungen für Standardisierung und Normung. Für den erfolgreichen Einsatz von IT-Plattformen benennen die Autoren abschließend relevante Handlungsfelder für verschiedene Akteure.

It is the aim of IoT-Testware to supply a rich set of TTCN-3 test suites and test cases for IoT technologies to enable developers in setting up a comprehensive test environment of their own, if needed from the beginning of a project. Initially, IoT-Testware will focus on protocols like CoAP and MQTT. To ensure test and implementation technology independence, the test suites will be realized in TTCN-3 and implemented with Titan. TTCN-3 has been defined and standardized by the European Telecommunication Standards Institute in ETSI ES 201873 and related extension packages. It is implemented and supported in Eclipse IoT by the Titan project. The test suites will contain tests for conformance, interoperability, robustness, and security aspects.

Vom Internet der Dinge (Internet of Things, kurz IoT) werden umfassende Weiterentwicklungen und Umbrüche erwartet, wie sie entlang der Verbreitung des »traditionellen« Internets in Kombination mit der Mobilkommunikation geschehen sind. So durchdringt das IoT alle Arbeits- und Lebensbereiche und liefert neue Ansätze für das Monitoring und die Steuerung der physischen Welt inklusive sowohl geschäftskritischer als auch sicherheitskritischer Systeme und Prozesse wie Energie- oder Verkehrsnetze. Daher sind die Qualitätsanforderungen an IoT-Komponenten und -Lösungen sehr hoch und müssen durch moderne Methoden zur Konstruktion und Absicherung der Qualitäten adressiert werden. Dieser Artikel gibt einen Überblick zum Verständnis von IoT und skizziert Qualitätsanforderungen und grundlegende Testansätze. Abschließend wird auf den beim ASQF und GTB in Entwicklung befindlichen Lehrplan zum IoT-Quality Engineering (IoT-QE) eingegangen.

Given that ICT is at the heart of today's Smart City approach, it is of paramount importance to investigate concepts, which would enable the unification, the common understanding and the replication of ICT architectures/solutions/models across multiple cities. This unified and replicable approach can be best achieved by a very abstract model, aiming to capture the taxonomy and high-level structure of complex integrative ICT solutions for Smart Cities. The approach should be based on the idea of openness with respect to interfaces, software components and especially data, which is to be seen as the main ingredient of an ICT eco-system for Smart Cities. This paper presents an Open Data based ICT Reference Architecture for Smart Cities, which is developed within the EU project Triangulum [1].

Mit urbanen Datenplattformen verbinden viele Kommunen und Städte einerseits ein noch schwer zu fassendes Konzept und andererseits die Möglichkeit für mehr Transparenz, Beteiligung als auch zusätzliche Einnahmen für die Haushaltskasse. Dieser Artikel beschreibt den Stand zu urbanen Datenplattformen und diskutiert das Pros und Cons direkter vs. indirekter Einnahmen aus öffentlichen Daten.

Das Internet der Dinge (Internet of Things, IoT) stellt Gesellschaft und Industrie vor große Herausforderungen. Ungeklärt ist beispielsweise, wie die mannigfaltigen Schnittstellen im IoT auch mit bestehenden IT-Systemen verbunden, genutzt und nicht zuletzt getestet und weiterentwickelt werden können. Auch die gesellschaftlichen Rahmenbedingungen wie Fragen der Sicherheit, des Datenschutzes und sogar der Ethik werden das Thema IoT formen. Um der Industrie Hilfe in Form von Methoden, Leitlinien zur Qualitätssicherung und Absicherung von IoT-Lösungen durch Qualifizierungsschemata und ein Glossar als De-facto-Standard anzubieten, hat sich ein Expertengremium führender Unternehmen in Deutschland gebildet, unter anderem mit DB Systel GmbH, SAP Deutschland und Atos Deutschland sowie mit namhaften Dienstleistern wie Sulzer GmbH, imbus AG, tecmata GmbH, sepp.med GmbH und dem Konsortium Testing4You.

Die Gigabitgesellschaft des Jahres 2025 ist eine fortgeschrittene Informationsgesellschaft, die vollständig von Informations- und Kommunikationstechnik durchdrungen ist. Mit dem Begriff Gigabitgesellschaft wird derzeit ein Zukunftsbild gezeichnet, in dem durch die fortschreitende Digitalisierung und intelligente Vernetzung neue Impulse für das Leben und Arbeiten in unserer Gesellschaft gesetzt werden. Dabei ist die grundsätzliche Bedeutung einer Breitbandinfrastruktur für Innovation und Wirtschaftswachstum allgemein anerkannt. Untersuchungsbedarf besteht jedoch mittel- bis langfristig insbesondere für die Anforderungen an die zukünftigen Netzinfrastrukturen von Industrie, öffentlicher Verwaltung und Haushalten in Deutschland sowie zu den technologischen Optionen zur adäquaten Adressierung dieser Bedarfe. Aus diesem Grund widmet sich diese Studie der Frage, wie sich eine Gigabitgesellschaft im Jahr 2025 definieren kann, welche Anwendungen das tägliche Leben der Menschen und die Wirtschaft 2025 prägen und welche Netzinfrastrukturen diese Anwendungen benötigen werden, um die Lebensqualität zu verbessern, die Wirtschaft in Deutschland zu stärken und neue Innovationspotenziale zu schaffen. Dabei müssen sowohl die Bandbreitenbedarfe dieser Anwendungen einzeln und zusammen sinnvoll abgeschätzt werden, als auch - aufgrund der neuen Anforderungen zukünftiger Anwendungen - die weiteren, über den reinen Datentransport hinausgehenden Eigenschaften für die Fortschreibung der Netzinfrastrukturen Berücksichtigung finden.

Gerade mit Blick auf die Digitalisierung in allen Lebens- und Arbeitsbereichen ist Softwarequalität zentraler denn je. Die beiden wichtigen Antipoden der Freiheit und Sicherheit in einer Gesellschaft werden zunehmend durch softwarebasierte Systeme be- einflusst, gar ermöglicht beziehungsweise verhindert. Dieser Artikel diskutiert Ansätze, wie softwarebasierte Systeme zuverlässiger, sicherer und nachvollziehbar entwickelt und betrieben werden können

Es gibt kaum noch Bereiche in unserem Alltag und Berufsleben, in denen uns nicht auf irgendeine Weise Software begegnet, uns begleitet oder bereits lenkt – denken Sie nur an die softwarebasierten Fahrfunktionen in Ihrem Auto. Dabei verlassen wir uns – bewusst oder unbewusst – darauf, dass die Software das tut, was sie tun soll: Wir erwarten eine angemessene Qualität dieser softwarebasierten Systeme. Und wir gehen davon aus, dass die Software von Experten ihres Fachs erarbeitet wurde.

This paper is about issues experienced along testing large-scale industrial products, with safety and security critical relevance. The challenges in testing - several thousand requirements for several product variants and various configurations - were addressed by test execution automation. However, since principal testing concepts as well as architectural concepts were ignored or poorly implemented, the test automation activities faced various difficulties within the considered projects. The current paper presents these issues in an abstracted manner and discusses possible solutions.

Along the increasing digitization and interconnection in almost every domain in society or business, data is growing exponentially. It is expected that the worldwide Internet traffic will triple until 2020 in comparison to 2015. In the same time, the transmitted data volume will move from 53,2 Exabytes per months to 161 Exabytes per months [Cisco, 2016]. Cities and communities can support the provisioning and usage of urban data and benefit from resulting new services for the monitoring, understanding, decision making, steering, and control. Providing urban data is also supported by the ongoing movement of opening governmental data, but goes beyond. Urban data can include data from public, industrial, scientific or private sources. Yet, the design of urban data is still ongoing and numerous initiatives and standardization efforts on smart cities and communities put the grounds for the uptake and interoperability of urban data.

Today's Network Management highly depends on manual configuration of networking devices. On one hand, this leads to a high number of configuration related errors in the network, which must be compensated with additional OPEX (operational expenditure) effort. On the other hand, Network Management is not able to cope with the growing size of networks and the variety of devices due to manual configuration. This paper applies concepts from the area of Autonomic Network Management in order to develop components and algorithms for the auto-configuration of OSPFv3 routing in IPv6 based fixed network environments. A reference model for autonomic networks is applied for the design of the components. The developed key functionalities include: the automated computation and dissemination of OSPFv3 protocol configurations, topology discovery, automated partitioning of the network into OSPF areas, and in-band/in-network configuration and management of OSPFv3 routing. The proposed concepts are evaluated based on a prototype implementation and on measurements of its performance in the testbed of a large scale European project.

Im diesjährigen Online Themenspecial Testing können wir wiederum interessante Beiträge präsentieren und damit zur Zuverlässigkeit und Sicherheit der Digitalisierung beitragen. Allerdings müssen wir die Test-Herausforderungen, die das kommende Internet-of-Things mit sich bringen wird, zukünftig noch stärker in den Blick nehmen.

Die Sicherung von Diensten spielt eine zentrale Rolle bei der Entwicklung einer sicherheitskritischen Infrastruktur. In diesem Artikel wird daher eine Integrierte Komponente zur Sicherung von Cloud Services (ISCS) präsentiert, die den Zugriff auf Dienste in der Cloud gewährleistet. Das bedeutet, dass die ISCS für Zugriffsaspekte wie Authentifizierung, Autorisierung und Registrierung zuständig ist. Für die ISCS wird OAuth und OpenID verwendet, welche mit bestehenden Open Source Bibliotheken implementiert werden. OAuth steht für einen Standard, welcher es Diensten und Applikationen erlaubt, über eine vertrauenswürdige Infrastruktur auf sicherheitskritische Ressourcen in der Cloud zuzugreifen. OpenID ist ein populärer Standard aus der Webcommunity, welcher die domainübergreifende Authentifizierung von Portalnutzern ermöglicht. Die Kombination beider Standards erlaubt das Angebot einer vielfältigen Funktionalität, welche die wesentlichen Bereiche der Sicherung von Cloud Services abdeckt.

A particular area of strength for the European industry is the development of value-added complex products such as aircrafts and automobiles. Challenges in coordination and information exchange as well as full verification and validation arise besides distribution. The internet of things gives the opportunity for solving the challenges by connecting designer, physical devices and models on the same level to the internet of systems engineering allowing exchangeability. In combination with distributed simulation and testing over the internet technological advantage is provided to the European systems integrator. This paper presents recent results of the European FP7 project SPRINT : Software Platform for Integration of Engineering and Things (SPRINT 2013). Grant Agreement No 257909

Overview on TTCN-3 concepts and syntax with examples, references to standards, and browseable grammar

Software testing has often to be done under severe pressure due to limited resources and a challenging time schedule facing the demand to assure the fulfillment of the software requirements. In addition, testing should unveil those software defects that harm the mission-critical functions of the software. Risk-based testing uses risk (re-)assessments to steer all phases of the test process to optimize testing efforts and limit risks of the software-based system. Due to its importance and high practical relevance, several risk-based testing approaches were proposed in academia and industry. This paper presents a taxonomy of risk-based testing providing a framework to understand, categorize, assess, and compare risk-based testing approaches to support their selection and tailoring for specific purposes. The taxonomy is aligned with the consideration of risks in all phases of the test process and consists of the top-level classes risk drivers, risk assessment, and risk-based test process. The taxonomy of risk-based testing has been developed by analyzing the work presented in available publications on risk-based testing. Afterwards, it has been applied to the work on risk-based testing presented in this special section of the International Journal on Software Tools for Technology Transfer.

An increasing number of datasets from government, public organizations and institutions are published as open data. Metadata that describes them, are cataloged at central places to enable a better access to these datasets. Quantifying the metadata quality can help to measure the efficiency of a catalog and discover low-quality metadata records which prevent the user from finding what she is looking for. We researched and implemented a range of metrics from the field of metadata quality assessment as part of an open data platform. This paper describes the platform that automatically assesses the quality of different open government data portals using the CKAN catalog software. The results are aggregated and visualized through a web application in order to establish a continuous and sustainable monitoring service.

Research and development for “smart cities” has been ongoing for almost a decade now. However, only now its rise can be seen in close relation to the newest developments in Information and Communication Technologies (ICTs) for mobile broadband communication, sensor technologies, machine-to-machine (M2M) communication, and efficient data and computing resources.

The idea of autonomic computing, and accordingly autonomic networking, has drawn the attention of industry and academia during the past years. An autonomic behavior is widely understood as a control loop which is realized by an autonomic entity/agent that manages some resources, in order to improve the performance and regulate diverse operational aspects of the managed network or IT infrastructure. Self-management, realized through autonomic behaviors, is an appealing and dangerous vision at the same time. On one hand, it promises to reduce the need for human involvement in the network and system management processes. On the other hand, it bears a number of potential pitfalls that could be even dangerous to the network, the IT infrastructure, and the corresponding services. One of these pitfalls is constituted by the stability of the control loops, and correspondingly by the interference among multiple autonomic agents operating in parallel. In this paper, a novel approach to ensuring runtime synchronization and stability of multiple parallel autonomic control loops is presented. We formally model the problem of runtime action synchronization, propose different possible solutions, and provide a case study, as well as different performance measurements based on a prototype that implements our approach.

A main requirement for the Future Internet is to enable self-management behaviors facilitating the network to adapt to changing conditions and self-heal in the face of erroneous states. On the basis of Autonomic Fault-Management principles, this paper proposes a framework consisting of a set of components operating inside the network elements and allowing the devices to collaboratively realize self-healing. In that context, Autonomic Fault-Management is intuitively constituted by the detection of the presence of faulty conditions, followed in turn by the self-diagnosis and identification of the corresponding root causes, and completed consequently by the removal of the identified root causes and their effects. The proposed framework implements a distributed control loop that interacts with the network operations personnel in case the current erroneous state is not resolvable by means of Autonomic Fault-Management. We argue that there are a number of mutual benefits between our proposed framework and the IPv6 protocol suite. This is demonstrated by a case study that illustrates these benefits and shows how the capabilities of IPv6 can be enhanced through the self-healing mechanisms of the proposed framework. Finally, the prototype implementation used for the case study is analyzed in terms of scalability and overhead produced in the network nodes.

In der Regel muss Software mit limitierten Ressourcen und unter hohem Zeitdruck getestet werden. Trotz dieser erschwerten Bedingungen besteht die Notwendigkeit sicherzustellen, dass die Software den an sie gestellten Anforderungen entspricht. Darüber hinaus müssen mögliche Fehler offengelegt werden, welche die missionskritischen Funktionen der Software gefährden. Dies kann nur mir Softwaretests geleistet werden. Risikobasiertes Testen (engl.: risk-based testing) fußt dabei auf dem Prinzip, dass Risiken bewertet und diese Bewertung kontinuierlich aktualisiert wird.

The ability to calculate hash values is fundamental for using cryptographic tools, such as digital signatures, with RDF data. Without hashing it is difficult to implement tamper-resistant attribution or provenance tracking, both important for establishing trust with open data.We propose a novel hash function for RDF graphs, which does not require altering the contents of the graph, does not need to record additional information, and does not depend on a concrete RDF syntax. We are also presenting a solution to the deterministic blank node labeling problem.

This overview article presents the Testing and Test Control Notation (TTCN-3) success story and serves as an introduction to this Special Section that contains five articles selected from the TTCN-3 user conference in 2011. The article sketches the development of TTCN-3 from its very beginning. It summarizes the current status of the language by reviewing its standardization process, available test suites, tools, and services as well as its training program. In addition, the article puts the articles selected for this Special Section into perspective, with regard to the evolution of TTCN-3 and the testing methodology in general. Last but not least, it discusses indicators for possible future developments of TTCN-3.

This overview article presents the Testing and Test Control Notation (TTCN-3) success story and serves as an introduction to this Special Section that contains five articles selected from the TTCN-3 user conference in 2011. The article sketches the development of TTCN-3 from its very beginning. It summarizes the current status of the language by reviewing its standardization process, available test suites, tools, and services as well as its training program. In addition, the article puts the articles selected for this Special Section into perspective, with regard to the evolution of TTCN-3 and the testing methodology in general. Last but not least, it discusses indicators for possible future developments of TTCN-3.

Das vorliegende Dokument ist dem Leitbild der acatech verpflichtet, in gesellschaftlicher Verantwortung wissenschaftsbasierte Handlungsempfehlungen zu erarbeiten. Daher wird angestrebt, das Thema Integrierende Informations- und Kommunikationstechnologien (IKT) für die Stadt der Zukunft zwar mit dem gegebenen Fokus IKT zu betrachten, IKT aber als Dienstleister für die Desiderate und Bedarfe zukunftsorientierter Städte zu sehen - und zwar aus dem Blickwinkel politisch Verantwortlicher ebenso wie aus dem von Wirtschaft sowie Bürgerinnen und Bürgern.

Open data has been given a lot of attention in the public. In some situation ‘open by default’ has become established as a core principle, whereas others argue about the limited results and the lack of robust studies demonstrating the value, and point to the risk that open data might turn out to be a short lived policy fad. This special issue contains a variety of research papers addressing this topic from different views and providing recent research results on open data. The papers in this issue deepen the understanding of open data and show that the subject of open data is moving from the general to the study of specifics. The special issue also includes invited papers presented at the first public meeting of the SharePSI project. Share-PSI 2.0 is the European network for the exchange of experience and ideas around implementing open data policies in the public sector.

In many development projects, testing has to be done under severe pressure due to limited resources, a challenging time schedule, and the demand to guarantee security and safety of the released software system. Risk-based testing, which utilizes identified risks of a software system for testing purposes, has a high potential to improve testing in this context. It optimizes the allocation of resources and time, is a means for mitigating risks, helps to early identify critical areas, and provides decision support for the management [1, 2].

Modern IP-based wide-area surveillance systems often build on networks of multi-modal, intelligent and mobile sensor units. Detection of complex events is performed on intelligent sensors and fusing input in the sensor units or centralized control room components. The domain of surveillance and public safety creates requirement for robustness and fault-tolerance. This article will present an automated intelligence architecture for mobile surveillance, which provides capabilities for combining on-board event detection in sensor units, centralized decision making on the server side, and automated exploitation of mobile surveillance unit positioning data. This architecture must be very reliable to provide services in the face of challenges such as natural disasters and fire, potentially damaging the infrastructure of the surveillance system. To increase its reliability and robustness, we study the introduction of a self-healing system into the architecture and examine the combined system’s operation in three case studies.

Recently, in the Security testing domain a lot of knowledge has been collected from a significant amount of research. The contribution provides an introduction to advanced security testing methods and techniques in the context of European research and standardization projects. This includes numerous guidelines and best practices that have been identified and are applied in the context of industrial case studies. In particular it addresses risk modeling, security test pattern, functional security tests as well as fuzz testing, as important contributions to systematic, automatized test approaches in research, industry and standardization.

Die Qualitätssicherung von vernetzten eingebetteten Systemen stellt auf Grund der immer größeren Bedeutung für Innovationen ein wichtiges Thema für die deutsche Industrie dar. In einer 2008 vom Bundesverband Informationswirtschaft, Telekommunikation und neue Medien (BITKOM) veröffentlichten Studie zur Bedeutung des Sektors Eingebettete Systeme wurde die Dienstleistungsbranche rund um dieses Themenfeld als Wachstumsträger der IT-Branche ermittelt. Dabei stellte der gestiegene Qualitätsanspruch an eingebettete Systeme gemeinsam mit der Reduzierung der Zeit bis zur Markteinführung eines Produktes (Time-to-Market) das wichtigste Thema für die befragten Unternehmen dar.

Von der Automatisierung von Testaufgaben durch softwarebasierte Werkzeuge wird bereits seit der Softwarekrise in den 1960er Jahren gesprochen. So formulierte z. B. Herbert D. Benington bei seinem Vortrag zum Symposium „Advanced programming methods for digital computers” des Navy Mathematical Computing Advisory Panels und des Office of Naval Research im Juni 1956 unter anderem: ,,After coding, each component subprogram is parameter tested on the machine by itself. This testing phase uses an environment that simulates pertinent portions of the system program. Each test performed during this phase is documented in a set of test specifications that detail the environment used and the outputs obtained (vgl. [Ben83]).``

The mobility data cloud is an Internet based platform for the integration of mobility related data from different data providers. It offers a framework for the acquisition, aggregation, provisioning, and analysis of mobility relevant data originating from various sources, e.g. Charging stations, gas stations, position of vehicles, battery charge condition of electric vehicles, utilization of vehicles, and data regarding public transport. These various types of data should facilitate the collaborative shared utilization of mobility resources such as electric vehicles, charging stations, etc. This paper presents the design and implementation of such an Internet based mobility data cloud. In addition, the presented prototype is evaluated based on trials in a European city.

Model-based testing is a recognized method for testing the functionality of a system under test. However, it is not only the functionality of a system that has to be assessed. Also the security aspect has to be tested, especially for systems that provide interfaces to the Internet. In order to find vulnerabilities that could be exploited to break into or to crash a system, fuzzing is an established technique in industry.Model-based fuzzing complements model-based testing of functionality in order to find vulnerabilities by injecting invalid input data into the system. While it focuses on invalid input data, we present a complementary approach called behavioral fuzzing. Behavioral fuzzing does not inject invalid input data but sends an invalid sequence of messages to the system under test. We start with existing UML sequence diagrams - e.g. functional test cases - and modify them by applying fuzzing operators in order to generate invalid sequences of messages. We present the identified fuzzing operators and propose a classification for them. A description of a case study from the ITEA-2 research project DIAMONDS as well as preliminary results are presented.

Real-time software is usually used in circumstances where safety is important and the margin for errors isnarrow. These kinds of systems have applicability in a broad band of domains as: automotive, avionics, airtraffic control, nuclear power stations, industrial control, etc. As the name denotes, the main feature of ``real-time'' applications is the criticality of their timeliness. Guaranteeing certain timeliness requiresappropriate testing. As manual testing is burdensome and error prone, automated testing techniques arerequired. Although the importance of having a standard environment for automatic testing is high, thetechnologies in this area are not sufficiently developed. This paper reviews the standardized testdescription language ``Testing and Test Control Notation version 3 (TTCN-3)'' as a mean for real-timetesting and proposes extensions to enable real-time testing with TTCN-3. The aim is to provide a completetesting solution for automatic functional and real-time testing, built around this already standardizedtesting language. The solution includes an environment for designing and running the tests written in theextended language. As a proof of concept, test examples, designed using the enhanced TTCN-3, are mappedto real-time platform implementations and the timeliness of each implementation is analyzed.

UML modeling environments for doing model-based testing are often not very comfortable to use and burden some knowledge about the internals of UML to the users, respectively test engineers. Test engineers, however, are seldom experts in UML, thus, the gain of efficiency model-based testing approaches entail, is reduced by a too generic tooling. The tool Fokus!MBT, developed by the competence center MOTION of Fraunhofer FOKUS, is a multi-paradigmatic test modeling environment based on the UML Testing Profile, an OMG-adopted industry-driven notation for model-based testing. Fokus!MBT simplifies the creation and authoring of test models with methodology-specific support. It is built on top of Eclipse Papyrus, a powerful open source UML modeling environment, which, in turn, relies on the Eclipse Modeling Framework and the Graphical Modeling Framework. This paper provides deep insights into the basic concepts and technical realization of Fokus!MBT as well as into the lessons we have learned during development and application.

A key challenge for open data portals is the aggregation of metadata from various data catalogs (on different administrative level or from different application fields) also known as metadata harvesting. This paper describes harvesting at the pilot of the German open government portal GovData.de, which is scheduled to become the data portal for all German public administration levels. At the launch of the pilot portal in February, eleven federal, state and local data catalogs were integrated, which produced about 2,000 open data sets. In the meantime, the number of data sets increased to over 3,100 mainly due to improved harvesting capabilities of the portal. This paper discusses GovData.de metadata schema and experiences with the different harvesting techniques that are in use at GovData.de: CKAN-Harvest, CKAN-API, CSW-Harvest and JSON-Dump.

Fuzz testing or fuzzing is interface robustness testing by stressing the interface of a system under test (SUT) with invalid input data. It aims at finding security-relevant weaknesses in the implementation that may result in a crash of the system-under-test or anomalous behavior. Fuzzing means sending invalid input data to the SUT, the input space is usually huge. This is also true for behavioral fuzzing where invalid message sequences are submitted to the SUT. Because systems are getting more and more complex, testing a single invalid message sequence becomes more and more time consuming due to startup and initialization of the SUT. We present an approach to make the test execution for behavioral fuzz testing more efficient by generating test cases at runtime instead of before execution, focusing on interesting regions of a message sequence based on a previously conducted risk analysis and reducing the test space by integrating already retrieved test results in the test generation process.

Current security testing is based mainly on audits of processes, systems and networks but this still lacks generic security models and systematic testing approaches that allow risk-oriented semi-automated analysis. The basic aim of the ITEA 2 DIAMONDS project was to produce an effective methodology capable of strengthening the practices of security testing commonly used in computer science and various industrial areas.

Von der Automatisierung von Testaufgaben durch softwarebasierte Werkzeuge wird bereits seit der Softwarekrise in den 1960er Jahren gesprochen. So formulierte z. B. Herbert D. Benington bei seinem Vortrag zum Symposium "Advanced programming methods for digital computers" des Navy Mathematical Computing Advisory Panels und des Office of Naval Research im Juni 1956 unter anderem: "After coding, each component subprogram is parameter tested on the machine by itself. This testing phase uses an environment that simulates pertinent portions of the system program. Each test performed during this phase is documented in a set of test specifications that detail the environment used and the outputs obtained."

The testing and test control Notation TTCN-3 language was created due to the imperative necessity to have universally understood language syntax able to describe test behavior specifications. TTCN-3 enables systematic, specification-based testing for various kinds of tests including, for example, conformance, interoperability, regression, robustness, performance, and scalability testing. A test system can communicate with a system under test (SUT) synchronously or asynchronously. In this chapter, a small test specification in TTCN-3 is given in order to give an insight into this technology. The semantics of TTCN-3 is defined by an operational semantics, which defines the meaning of TTCN-3 behaviors in an intuitive and unambiguous manner. The chapter also describes different case studies that demonstrate the application of the TTCN-3 to test various target systems.

Requirements on the quality of open data depend on the exact context of a software and has to be set specifically for the different kinds of data in software-based systems. Quality of open data characteristics according to ISTQB include aspects of : currency; relevance; consistency; reliability; correctness; and completeness.

Nowadays, software-intensive systems continuously pervade several areas of daily life, even critical ones, and replace established mechanical or manual solutions. Development and quality assurance methods have to ensure that these software-intensive systems are delivered both with adequate quality, optimized resources and within the scheduled time frame. The idea of risk-based testing is to prioritize testing activities to what is deemed critical for the software-intensive system. Although there is a common agreement that risk-based testing techniques ought to be rigorously applied, especially for safety- and security-critical systems, there is actually little knowledge available on how to systematically come to risk-optimized test suites. This paper presents a novel approach to risk-based testing that deals with the transition from risk management and requirements engineering to test design activities and test case generation by using models. The main contribution of the paper is the description of a methodology that allows an easy combination of test generation directives and risk level in order to generate risk-optimized test suites.

Der freie Zugang zu öffentlichen Verwaltungsdaten - lesbar für Mensch und Maschine - ist ein wichtiger Grundstein intelligenter Städte. Transparente Demokratie, wirtschaftliche Wertschöpfung und effiziente Kommunikation sind die Motivatoren für die Entwicklung urbaner Datenplattformen. Wie ein solches Projekt starten kann, zeigt das Berliner Portal für offene Daten daten.berlin.de.

Autonomic Networking is seen as one of the hopes to improve the Quality of Service (QoS) provided to end users in the Internet. Especially in the context of Video on Demand (VOD), content and network providers have to deal with the dynamics of network and server loads which can severely impair the services provided to end users. Our recent research efforts seek to provide a remedy for this issue by introducing an autonomic framework for optimal network resource utilization that allows content and network providers to cooperate, and take into account aspects of the network state as well as server loads while optimizing the services for their subscribers. The framework is specified, designed, and implemented in an IPv6 testbed that was setup by the participating partners in the course of a large scale European project. Finally, a scenario realized in this testbed is presented that illustrates the benefits of the proposed approach.

Due to limited test resources, it is often necessary to prioritize and select test cases for a given system under test. Although test case prioritization is well studied and understood, its combination with test data generation is difficult and not completely solved yet. For example, the Classification Tree Method is a well established method for test data generation, however the application of prioritization techniques to it is a current research topic. We present an extension of the classification tree method that allows the generation of optimized test suites, containing test cases ordered according to their importance with respect to test goals. The presented algorithms are incorporated into the Classification Tree Editor and empirically evaluated on a set of benchmarks.

Although security and model-based testing are not new areas of research, they are still under development and highly relevant. In particular, their combination is a challenge for academic work and industrial applications. Some examples of systematic and automated security testing include: security functional testing, model-based fuzzing, risk-oriented testing and the usage of security test pattern.

Offene Daten (auf englisch: Open Data) sind einer der wichtigsten IT-Trends der letzten Jahre, mit dem das Ziel verbunden wird, den freien Austausch und die Weiterverwendung von Daten und darauf aufbauender Mehrwert-Angebote zu erleichtern. Die bestehenden Informations- und Datenportale von Bund, Ländern und Kommunen in Deutschland sind eine gute Ausgangsbasis für den weiteren Ausbau des Angebots offener Daten in Deutschland. Politik, Verwaltung, Wirtschaft und Wissenschaft haben mit der »Dresdner Vereinbarung zum 5. Nationalen IT-Gipfel« im Dezember 2010 als einen nächsten Schritt bis 2013 den Aufbau einer zentral zugänglichen, den Interessen der NutzerInnen an einer einem einheitlichen, leichten und benutzerfreundlichen Zugriff gerecht werdenden Open-Government-Plattform mit dem Schwerpunkt auf der Bereitstellung offener Verwaltungsdaten vereinbart. Um die Aufbereitung der und den Zugang zu den Daten zu lenken und zu erleichtern, wird derzeit ein Metadaten-Schema erarbeitet, das in seinem initialen Entwurf hier vorgestellt wird.

Open Data (OD) is an emerging trend that aims to facilitate the freedom and reuse of information. Therefore, tools, applications and platforms are required that enable the publishing and consumption of data. In this paper, we present our experience from the integration of components that should constitute an OD platform. The proposed solution is able to store datasets as linked data, to catalogue the datasets, and provides a portal with features for supporting community activities. Further, we exemplify the utilization of the proposed platform, by describing a touristic city guide web mashup that uses published Open Data. This application consumes data in machine readable format over APIs provided by the OD platform.

The European ITEA2 project DIAMONDS (Development and Industrial Application of Multi-Domain Security Testing Technologies) develops under the direction of Fraunhofer FOKUS, Berlin efficient and automated security test methods for security-critical, networked systems in various industrial domains such as industrial automation, banking and telecommunications. DIAMONDS develops methods to design objective, transparent, repeatable, and automated security tests that focus on system specifications and related risks. The project goals include the development of a security test pattern catalogue and the development of model-based security testing techniques such as risk-based testing and model-based fuzz testing. The project results are made available through publications and contributions to the standardization at ETSI and other standardization bodies. The presentation focusses on model-based fuzz testing, reviews the state of the art, compare it to similar approaches such as mutation testing, and presents first results on behaviour fuzzing for security testing.

Security testing aims at validating software system requirements related to security properties like confidentiality, integrity, authentication, authorization, availability, and non-repudiation. Although security testing techniques are available for many years, there has been little approaches that allow for specification of test cases at a higher level of abstraction, for enabling guidance on test identification and specification as well as for automated test generation. Model-based security testing (MBST) is a relatively new field and especially dedicated to the systematic and efficient specification and documentation of security test objectives, security test cases and test suites, as well as to their automated or semi-automated generation. In particular, the combination of security modelling and test generation approaches is still a challenge in research and of high interest for industrial applications. MBST includes e.g. security functional testing, model-based fuzzing, risk- and threat-oriented testing, and the usage of security test patterns. This paper provides a survey on MBST techniques and the related models as well as samples of new methods and tools that are under development in the European ITEA2-project DIAMONDS.

Model-based testing (MBT) strives to automatically and systematically generate test cases. In this column, Ina Schieferdecker introduces MBT technologies and methods.

Quality of life in an urban environment depends strongly on ecological, social and mobility aspects. A major innovation in that context is given by the emergence of electric vehicles. Additionally, the explosive growth of social networks has shown how the Internet can be used to maintain and create communities, thereby bringing mutual benefits to the involved participants. Combining both, there is an obvious potential for the realization of collaborative electric vehicle sharing within a city. In this paper, we investigate one of the key aspects required to realize the vision of electric vehicle sharing - a cloud infrastructure for handling the required data. We propose a distributed architecture for the realization of such data cloud. Further, we demonstrate how ISP networks and the electric mobility data cloud can collaborate in order to provide efficient streaming of continuous data.

A compelling direction of improving trustworthiness of software-based systems is to open their ingredients: Open source software, open data sets, open system interfaces like open technical standards allow constructing major or up to all elements of a software-based systems. That gives means to use the wisdom of the crowd to assess and evaluate the quality, security and trustworthiness of software components. In addition, the software components can mature along a continuous feedback and revision loop with the crowd.

Designing and developing complex multi-domain systems such as autonomic systems and networks is not only a creative but also a systems engineering challenge. Current systems engineering methodologies are unsuitable, as autonomics is a multi-domain field that requires the knowledge of several domains, such as control-theory, computer networks, software-engineering, etc., and comes with its own set of requirements and challenges. We showcase why and how current systems engineering approaches fail, and advocate for an avant-garde systems engineering approach to designing safety-critical and multidomain systems, such as autonomic entities. In this paper, we introduce a new model-driven systems engineering methodology, called the Rotary Dial Model (RDM), whose functioning is inspired by the standard telephone rotary dial. We describe in detail the working of the RDM model, and showcase its robustness and flexibility for the system design process.

Diese Frage wird mir immer wieder und aus verschiedenen Blickwinkeln gestellt: Wie können nachhaltig Verbesserungen für den Testprozess und sein Management realisiert werden, kann die Integration der Systementwicklung mit der Qualitätssicherung gestärkt werden, können passende Werkzeuge und Abläufe gefunden werden, können Tester auf neue Aufgaben vorbereitet werden? Sie wird auch oftmals von Forschern gestellt, die aktuelle Fragestellungen für Forschungsprojekte und Promotionen suchen. Die Frage wird auch vor dem Hintergrund gestellt, dass sich an den Testmethoden im Vergleich zur Softwarekrise in den 60ern – nun auch schon 50 Jahre her – wenig geändert hat und dass die Testkosten wie die Restfehlerraten auf hohem Niveau verharren.

This chapter provides a taxonomy of Model-based Testing (MBT) based on the approaches that are presented in this book as well as in the related literature. The techniques for testing are categorized using a number of dimensions to familiarize the reader with the terminology used in the chapters that follow.

Advanced methods and tools for GUI software development allow a rapid and iterative process of prototyping and usability testing. Unfortunately, even with the support of test automation tools, testing of GUI software requires a lot of manual work, especially when the application under test is changing rapidly. In this paper we present an improved method and tool support for automated test modeling of Java GUI applications for model-based testing (MBT) purposes. The implemented GUI Driver tool generates structural models combined with a GUI state model presenting the behavior of the GUI application that is executed and observed automatically. The GUI Driver tool is combined with an open source MBT tool to form a tool chain to support automated testing of Java GUI applications. The models generated by the GUI Driver are used to generate test sequences with MBT tool, and the test sequences are then executed with the GUI Driver to generate a test report.

Zum 15. Jubiläum des Arbeitskreises Software-Qualität und -Fortbildung (ASQF) wurde am 24. September 2011 erstmalig der deutsche Preis für Software-Qualität an Harry M. Sneed für sein Lebenswerk verliehen. Harry M. Sneed bearbeitet seit 35 Jahren das Thema der Software-Qualität, nicht nur, aber immer wieder auch als Software-Tester. Dabei hat er bereits Ende der 70er und mittlerweile gut 20 Testwerkzeuge konzipiert, umgesetzt und angewendet, eine Leidenschaft, die ihm noch heute immer wieder neue Einsichten in Testmethoden gibt, mit allen nötigen als auch möglichen Erweiterungen.

Im Auftrag des Landes Berlin untersuchte das Fraunhofer-Institut für Offene Kommunikationssysteme FOKUS im Rahmen des E-Government-Programms "ServiceStadt Berlin" die organisatorischen, rechtlichen und technischen Grundlagen für Open Data in der Hauptstadt. Hierbei wurde eine Konzeptstudie erarbeitet und in einem Online-Pilotsystem umgesetzt, dem ersten Open Data Portal in Deutschland. Diese Studie dokumentiert neben den Rahmenbedingungen das Konzept zur Umsetzung, den Prototyp sowie erste Erfahrungen aus dem Pilotbetrieb.

Existing online social networks hardly care about users’ privacy rights. In particular, they do not permit users to keep control over “their” data. By “their” data, we denote data that refers to the respective user as an identifiable object within (textual, audio, image or video) media. The well-known concept of “usage control” employs a usage rights’ perspective (e.g. DRM), but it does not explicitly deal with privacy. In this paper, we instead propose the concept of “data control”, which exactly focusses on privacy rights and therefore employs a control rights’ perspective. Based on data control, we propose a defamation-free network (DFN) in which control rights are not only manifest and visible, but can also be exercised. We examine the main usage scenarios of such a network, and discuss the possible approaches for implementing it. Finally, we sketch a solution with an underlying P2P architecture and highlight the basic technological challenges and requirements.

The term dependability was defined in the 1980s to encompass aspects like fault tolerance and system reliability. According to IFIP, it is defined as the trustworthiness of a computing system which allows reliance to be justifiably placed on the service it delivers. Hence, dependability is the capability of a system to successfully and safely complete its mission. This chapter concentrates on safety and reliability aspects. It starts with a review of the basic terminology including, for example, fault, failure, availability, and integrity. In the following, a mathematical model of fault-tolerant systems is defined. It is used in the further sections for comparison with different techniques for safety and reliability analysis. Also selected currently available model-based development tools are reviewed. A summary and identification of future research challenges conclude the chapter.

Software-driven electronic control units (ECUs) are increasingly adopted in the creation of more secure, comfortable, and flexible systems. Unlike conventional software applications, ECUs are real-time systems that may be affected directly by the physical environment they operate in. Whereas for software applications testing with specified inputs and checking whether the outputs match the expectations are in many cases sufficient, such an approach is no longer adequate for the testing of ECUs. Because of the real-time requirements and the close interrelation with the physical environment, proper testing of ECUs must directly consider the feedback from the environment, as well as the feedback from the system under test (SUT) to generate adequate test input data and calculate the test verdict. Such simulation and testing approaches dedicated to verify feedback control systems are normally realized using so-called closed loop architectures (Montenegro, Jähnichen, and Maibaum 2006, Lu et al. 2002, Kendall and Jones 1999), where the part of the feedback control system that is being verified is said to be “in the loop.” During the respective stages in the development lifecycle of ECUs, models, software, and hardware are commonly placed in the loop for testing purposes.

Modelling system behaviour by means of UML Behavioral State Machines is an established practice in software engineering. Usually, code generation is employed to create a system's software components. Although this approach yields software with a good runtime performance, the resulting system behaviour is static. Changes to the behaviour model necessarily provoke an iteration in the code generation workflow and a re-deployment of the generated artefacts. In the area of autonomic systems engineering, it is assumed that systems are able to adapt their runtime behaviour in response to a changing context. Thus, the constraints imposed by a code generation approach make runtime adaptation difficult, if not impossible. This article investigates a solution to this problem by employing interpretation techniques for the runtime execution of UML State Machines, enabling the adaptability of a system's runtime behaviour on the level of single model elements. This is done by devising concepts for behaviour model interpretation, which are then used in a proof-of-concept implementation to demonstrate the feasibility of the approach. For a quantitative evaluation we provide a performance comparison between the proof-of-concept implementation and generated code for a number of benchmark models. We find that UML State Machine interpretation has a performance overhead when compared with static code generation, but found it to be adequate for the majority of situations, except when dealing with high-throughput or delay-sensitive data.

Requirements engineering is vital for a software development project's success or failure. As today's software systems are getting more and more complex, their related requirements specifications contain often hundreds, even thousands of natural language requirements. The so called behavior engineering developed by Geoff Dromey is suitable to handle the complexity of large-scale software requirements specification by relying on a scalable requirements formalization methodology. The outcome of that methodology is a requirements model in the behavior tree notation, describing the intended, externally visible behavior of the system. By deliberately extending the behavior engineering methodology with testing activities, those requirements models can be further exploited for testing purposes like system and acceptance level testing. This also addresses the common challenge in model-based testing scenarios, namely the availability of a meaningful test model. By reusing a testable requirements model, both the system and test model can be derived from the same specification, since all requirements are intended to be captured in the requirements model properly and consistently. In this paper, we present an approach of how behavior trees can be extended with testing activities to leverage the definition of test requirements. We also briefly discuss how augmenting test-related information to make the requirements model more complete in terms of the IEEE830 standard.

Die weltweite Urbanisierung ist eine zentrale Herausforderung des 21. Jahrhunderts. Seit 2007 wohnen erstmals mehr Menschen in Ballungsräumen als in ländlichen Regionen, im Jahr 2030 werden es 60 Prozent sein. Umso notweniger sind Städte und Agglomerationen auf vernetzte Technologien angewiesen. „Smarte“ Technologien bieten hier Antworten in unterschiedlichen Bedarfsfeldern. Gerade Deutschland hat gute Chancen, sich als Leitanbieter innovativer Gesamtlösungen zu etablieren – wenn dafür die Rahmenbedingungen geschaffen werden. Die vorliegende Stellungnahme entstand in Vorbereitung des ersten „Innovationsdialogs zwischen Bundesregierung, Wirtschaft und Wissenschaft“ im September 2010. Sie stellt aber eine eigenständige Veröffentlichung der Deutschen Akademie der Technikwissenschaften dar. Darin werden Handlungsempfehlungen unterbreitet, wie der Bedarf an integrierten Lösungen neue Wertschöpfungsmöglichkeiten erschließen kann.

Das UML Testing Profile [UTP] ist eine grafische Modellierungssprache zur Definition, Dokumentation, Visualisierung, Implementierung und zum Austausch von modellbasierten Testspezifikationen auf Basis der UML [UML]. Der normative Teil der Spezifikation definiert das native UML-Profil des UTP(1). Die Spezifikation des UTP als UML-Profil ist im Vergleich zu anderen Sprachen der UML-Familie [SysML, MARTE, SoaML] deutlich schlanker und kompakter, erfordert aber gleichzeitig profundes UML-Wissen, um die Potenziale von UTP voll ausschöpfen zu können. Ein Umstand, der die Akzeptanz des UTP bis heute behindert, da nur wenige Tester in den Konzepten der Modellierung im Allgemeinen ausreichend geschult und mit der UML im Besonderen vertraut sind. Dieser Artikel widmet sich daher neben einer kurzen Einführung in das UTP vor allem Fragen seiner Anwendbarkeit. Dabei stehen weniger spezifische Domänen, sondern eher grundsätzliche, testmodellspezifische Fragestellungen im Vordergrund.

The adoption of standards in eHealth domain, such as Health Level 7 (HL7) used for data representation, and Integrating Healthcare Enterprise (IHE) for describing interactions between actors, is an important step in enabling healthcare systems to interoperate with each other. This paper addresses the challenges of interoperability testing of different HL7 based systems and presents a test framework based on TTCN-3 language.

One of the challenges of evolving healthcare information systems used in healthcare domain is their interoperability. Interoperability not only permits healthcare institutes to take advantage of using heterogeneous solutions from different vendors but it also offers the basis for creating a larger range of services at the edge of medicine/healthcare and information technology. New services can be created in a consistent way based on common approaches of structuring and representing healthcare data. This paper presents a test framework for testing interoperability of eHealth systems. As a major result, the test framework is designed for test automation and extendability with respect to test configurations and test cases.

Das europäische Forschungsprojekt D-MINT wurde in der Zeit vom 01.04.2007 bis 31.12.2009 finanziell gefördert durch das Bundesministerium für Bildung und Forschung. Dieser Bericht beschreibt das Projekt und fasst insbesondere die erreichten Ergebnisse und den Nutzen bei den beteiligten deutschen Projektpartnern zusammen.

The evolution of the World Wide Web has created a new environment where people can interact, e.g. talk to their friends, shop online, conduct business meetings, etc. Trust and trustworthiness are central notions in human interaction; in particular, they represent important criteria for every Internet user because of the multitude of choices they are faced with when choosing whom to interact with. This paper presents a simulation model implemented in Ptolemy II for the simulative analysis of trust policies in networked environments, called communication space (CS). The model reflects both the CS structure e.g. principals, roles, and event structure, and the interactions between the elements of CSs. Principal behavior is based on Markov chains and on criteria for selecting peers that initiate transactions. We investigate the efficiency of trust policies based on local observations and evaluation of interactions by examining a case study based on the popular auction site eBay.

Embedded systems pose special challenges to system evolution: they're embedded in a changing environment, often interacting with evolving processes of human organizations, and thus must be verified because of their critical nature. Complicating the situation, the analyses and testing regimens used to verify them must evolve as well. Both software engineering research and industrial practice need to improve to address these problems. While admittedly underemphasized in software engineering education, system evolution is crucial, and the challenges discussed here will be addressed by improving on the initial results presented.

Model-Based Testing (MBT) constitutes a number of technologies, methods, and approaches, with the aim of improving the quality, efficiency, and effectiveness of test processes, tasks, and artifacts. Started as a pure academic field of application, it has gained significance for industrial domains in recent years. Moreover, the ongoing adoption of Model-Based Engineering techniques by industrial-grade software engineering companies provides a solid basis to introduce and apply MBT approaches and hopefully will lead to a larger acceptance of model-based techniques. This entry presents background information on testing in general, motivates the application of MBT approaches in particular, and reviews related work and standards.

Model-based testing (MBT) constitutes a number of technologies, methods, and approaches with the aim of improving the quality, efficiency, and effectiveness of test processes, tasks, and artifacts. The entry investigates MBT approaches and the languages and notations used. It is the successor entry to model-based testing by the same authors.

Model-Based Testing (MBT) constitutes a number of technologies, methods, and approaches, with the aim of improving the quality, efficiency, and effectiveness of test processes, tasks, and artifacts. This entry examines the use of MBT to solve problems and answer challenges in the field of test automation, test quality improvement, and test efficiency. It reviews the application of MBT in different domains and trends and challenges for the further development of MBT approaches. Subsequently, tools supporting MBT processes are presented and a discussion on the MBT return of investment is given.

Nachdem modellbasierte Entwicklungsmethoden in den letzten Jahren zunehmend Einzug in die Entwicklungsabteilungen gehalten haben, ist das modellbasierte Testen (MBT) verstärkt in den Fokus von Forschung und Industrie gerückt, um die Vorteile der modellbasierten Softwareentwicklung auf den Testprozess zu übertragen. Nach [Utting] steht das klassische Testen immer noch vor denselben Problemen wie vor zehn Jahren: die Systematik bei der Erhebung von Testfällen ist oftmals nicht nachvollziehbar, wodurch sich der Gesamtprozess nicht reproduzieren lässt; die Dokumentation des Testprozesses ist oft mangelhaft oder gar nicht vorhanden, was beim Ausscheiden eines Testers zu signifikantem Verlust von Wissen führt; oft erfährt der Beruf des ,,Testers`` generell eine zu geringe Wertschätzung. Nach Uttings Einschätzung (sowie zahlreicher weiterer Experten) ist MBT ein viel versprechender Ansatz, um einige dieser Probleme zu lösen, mindestens jedoch deutlich abzuschwächen.

Hand aufs Herz: mit dem Software-Testen ist es wie mit gesunder Lebensweise: eigentlich wissen wir, was zu tun wäre, doch fällt die Umsetzung aus verschiedensten Gründen schwer und wird noch allzu oft anderen (an- bzw. vorgeblichen) Dringlichkeiten geopfert. Nun wollen wir hier nicht mit erhobenem Zeigefinger auf die Notwendigkeit nachhaltiger Prozesse bei der Qualitätssicherung eingehen, sondern Ihnen nützliche Tipps und Hinweise geben und Ihnen behilflich sein, den Stand Ihres Software-Testens verbessern zu können. Dieser Aufgabe haben wir uns seit 2008 verschrieben und freuen uns, Ihnen nun die dritte Ausgabe des OBJEKTspektrum Online Themenspecials über Software-Testen vorstellen zu können.

Test automation encompasses all activities to automate various steps in the overall testing process including automation of test management, test generation, or test execution. The standardized Testing and Test Control Notation (TTCN-3) addresses selected challenges by defining a test specification language and a test system architecture that enables the implementation and execution of TTCN-3 test suites. Over the years, the standard has continuously being maintained and evolved. For example, concepts for static test configurations or for advanced parameterization and typing have been defined. The paper reviews the history and current status of TTCN-3 and concludes by giving an overview on recent extensions of TTCN-3 and future plans.

In the engineering world, more and more accent is put on the real-time embedded applications. They are used almost everywhere, in domains as: automotive, robotics, health, avionics, spacecraft, industrial control, etc.. Often, on the proper functionality of real-time systems, depends the well functioning of a whole system and their mistakes might have fatal consequences. Often such real-time applications have additional requirements for a fast and reliable communication with the rest of the system they are part from. In order to prohibit misbehavior of such a critical application, proper tests should be developed and applied to it. Although real-time application development gained so much ground and a lot of tools and paradigms for real-time emerged, there are still little effort invested in the development of tools for software real-time testing. In this paper we propose a testing framework solution for real-time, based on a standardized testing language, that was used with success by now in many domains, as telecommunications, automotive, health, etc.. This language is named TTCN-3 - the acronym comes from Testing and Test Control Notation - and it was developed by European Telecommunications Standards Institute (ETSI). The purpose of this paper is to demonstrate that the proposed test framework achieves both a time deterministic behavior and high performance at the communication points with the System Under Test(SUT).

This contribution addresses systematic test development methods to include an algorithm to retrieve a test suite execution control in order to run test cases with high priority earlier than others. The approach uses a model that allows both the introduction of user-defined weightings for system features within the test model and an automatic calculation of the test ordering. Based on an algorithm for the calculation of test case weights first results from the application of a tool implementation in pilot projects have been described.

Patterns have been successfully applied in product software development to improve the development process, by facilitating reuse, communication and documentation of sound solutions. However, the testing domain is yet to benefit from a similar approach. This although, identifying and instrumenting patterns in test modelling to facilitate reuse appears to be a promising approach for reducing the test development cycle. This paper presents a pattern language for test modelling, i.e. a collection of patterns and good practices known for improving the test development process towards more efficiency and shorter development cycles. Further, the paper presents first experiences on applying those patterns in a case study.

Choreography models describe the communication protocols between services. Testing of service choreographies is an important task for the quality assurance of service-based systems as used e.g. in the context of service-oriented architectures (SOA). The formal modeling of service choreographies enables a model-based integration testing (MBIT) approach. We present MBIT methods for our service choreography modeling approach called Message Choreography Models (MCM). For the model-based testing of service choreographies, MCMs are translated into Event-B models and used as input for our test generator which uses the model checker ProB.

The introduction of open and standardized service delivery platforms (SDPs) is a key element towards the highly anticipated convergence of Internet and telecommunication networks. SDPs provide applications with a middleware in the form of open APIs or Web services to facilitate access to heterogeneous network infrastructure in a common manner. However, new challenges have emerged regarding the robustness of such platforms against potential security threats and other types of failures which could affect service availability. Current approaches in testing Web services target stateless unit-level testing and tend to focus on compliance of SOAP/XML data to the Web service's WSDL schema. We propose to address service level tests by a pattern-driven test modeling approach which allows a stateful testing on service level and which reduces the development cycle of SOA tests. The approach is built around a UML meta-model embodying test patterns, i.e. recurrent solutions in specifying tests. This allows to specify test requirements based on known vulnerabilities and attack patterns at the same level of abstraction as the services under test, so that the focus can be put on availability aspects. Using examples from Parlay-X Web services, we demonstrate how TTCN-3 test scripts can be generated automatically from the abstract test models to facilitate test deployment.

Ein abgestürztes Office-Programm ist nur ärgerlich. Ein kollabiertes Gepäcksystem wie 2008 im neuen Terminal des Londoner Großflughafens Heathrow hingegen kostet Millionen. Fehler in IT-Systemen können weitreichende Folgen haben. Solchen Pannen sollen Software-Tester vorbeugen.

With the gap between manufacturers and clinical environments widening, integration of complex medical IT systems into heterogeneous clinical networks becomes increasingly challenging. This needs to be addressed says Prof Dr -Ing Ina Schieferdecker and Dr Armin Metzger of the ReTeMeS project.

Although there are many test systems and languages which were designed and tailored specifically for the Web, the versatility of standardized test languages makes them good candidates for such testing activities. TTCN-3 is a standardized test language and test architecture for distributed reactive black-box testing. In this paper, we present a framework for testing Web applications and Web services using TTCN-3 as language of choice. Besides this abstract test layer, we have developed a test adapter and codec that permit seamless integration and usage of browser simulators while providing for functional testing as well as non-functional testing (e.g. distributed load-testing, security testing). The capabilities were proved for certain proof-of-concept web applications; a case study illustrates the relevance of the framework in the context of quality assessment and testing.

Research in the area of Self-Managing Networks is on the rise. Some research initiatives are calling for cleanslate/revolutionary designs of future networks (e.g. the future internet) while some initiatives are calling for an evolutionary approach and are focused on making today’s networks evolve by introducing new communication paradigms into the network models and architectures in an incremental way. For both types of initiatives towards the design of future networks, one of the most important questions being raised is: “How can the well known and very successful FCAPS network management framework be adopted and extended as necessitated by emerging network architecture designs for Self-Managing Networks?”. Recently, a framework called UniFAFF has emerged, that answers the posed question with respect to moving from today’s Fault-Management whose processes are not autonomic, to making Fault-Management processes become autonomic as necessitated by Self-Managing Networks. UniFAFF stands for Unified Framework for Implementing Autonomic Fault-Management and Failure-Detection for Self-Managing Networks. This paper reports on the first attempt to implement the UniFAFF framework for a clean-slate type of network design — namely the ANA network architectural design.

The success of service-oriented architectures (SOA) depends on faultless and seamless service integration. Formal modeling of global communication protocols between services enables a model-based integration testing (MBIT) approach. In this paper we present an MBIT approach based on SAP proprietary choreography models called message choreography models (MCM). We explain how MBIT fits into the SAP testing methodology for SOA and give some insights into the experience we gained from the work.

The purpose of this chapter is to introduce the test methods applied for embedded systems addressing selected problems in the automotive domain. Model-based test approaches are reviewed and categorized. Weak points are identified and a novel test method is proposed. It is called model-in-the-loop for embedded system test (MiLEST) and is realized in MATLAB®/Simulink®/Stateflow® environment. Its main contribution refers to functional black-box testing based on the system and test models. It is contrasted with the test methods currently applied in the industry that form dedicated solutions, usually specialized in a concrete testing context. The developed signal-feature-oriented paradigm developed herewith allows the abstract description of signals and their properties. It addresses the problem of missing reference signal flows and allows for a systematic and automatic test data selection. Processing of both discrete and continuous signals is possible, so that the hybrid behavior of embedded systems can be handled.

A system for automatically testing a model of system under test includes (a) means for the automatic generation of a test harness; (b) means for automatic generation of test specifications based on the analysis of the results obtained from the simulation of the test harness; (c) means for the automatic generation of test data and test controls; (d) means for automatic evaluation of the test quality and the automatic generation of a verdict. A method for automatically testing a model system includes (a) detecting at least one feature of an input signal to the model system and (b) detecting at least one feature of an output signal of the model system, wherein the at least one feature of the input signal or the input signal is generated automatically depending on at least one feature of the at least one output signal.

Service-Oriented Architecture (SOA) constitutes a modern, standards-based and technology-independent paradigm and architectural style for distributed enterprise computing. The SOA style promotes the publishing, discovery, and binding of loosely-coupled, network-accessible software services. With SOA systems operating in distributed and heterogeneous execution environments, the engineers of such systems are confined by the limits of traditional software engineering. In this position paper, we scrutinize the fundamental tenets underpinning the development and maintenance of SOA systems. In particular, we introduce software service engineering as an emerging discipline that entails a departure from traditional software engineering disciplines, embracing the dasiaopen world assumptionpsila. We characterize software service engineering via seven defining tenets. Lastly, we survey related research challenges.

Deutschland ist bekannt für seine qualitativ hochwertigen Produkte. Über Jahrhunderte haben sich rigide Qualitätsvorschriften in den Ingenieursdisziplinen entwickelt, welche über Standards gefordert und durch verschiedene Abnahmeverfahren (z.B. den TÜV) geprüft werden. Bei der Software sind wir mitnichten soweit. Es wird zwar verstanden, dass dieser immaterielle Teil der Produkte maßgeblich zur Funktionalität der Produkte beiträgt (so ist beispielsweise Software in der Automobilelektronik mittlerweile der Innovationstreiber), doch ist die Qualitätssicherung der Software noch nicht im gleichen Maße durch Standards unterlegt wie in den anderen technischen Bereichen.

Patterns have been successfully applied in software development to improve the development process, by facilitating reuse, communication and documentation of sound solutions. However, the testing domain is yet to benefit from a similar approach. This although, with the growing complexity of test automation solutions, identifying and instrumenting patterns in test design to facilitate reuse appears to be a promising approach for shortening the development cycle and save costs. This paper presents a collection of patterns for designing test automation solutions for reactive software systems and reports on first experiences of applying those patterns in a case study.

The problems of testing software based systems that, like automobiles, steadily increase in complexity are still not solved. To cope with the requirements and complexities of today's systems, adequate test solutions are needed, which at least feature a minimum of flexibility, reusability and abstraction. The Testing and Test Control Notation TTCN-3 is a test specification language, which was originally developed to meet the requirements of testing telecommunication systems. The language is modular, well-structured, standardized and supports testing of communicating systems. However, the correctness of a large number of embedded systems can not be assessed by checking functional requirements only. In addition to that non-functional requirements, especially time related input-output behavior, have to be considered. The current version of TTCN-3 has only limited capabilities for testing such non-functional properties. To overcome these limitations we will extend TTCN-3 with a small set of specific language means that are dedicated to check real-time properties of embedded system. We will explain the syntax and semantics of the new constructs, compare our solution with the expressiveness of standard TTCN-3 and, as a proof of concept, provide a small example from the automotive domain that particularly motivate the use of TTCN-3 in the context of AUTOSAR.

Component integration plays a decisive role in service-oriented architectures (SOAs). The technical implementation must faithfully reflect business and enterprise integration requirements. This implies a good understanding of the globally observable message choreography but also of how messages are handled by the involved components and by the SOA middleware. In this paper, we present a solution to the problem of keeping global and local viewpoints in synchronization via a common message choreography metamodel. As main contribution we shape various interpretations of global choreographies, which were left unspecified in state-of-the-art choreography approaches. We have implemented a message choreography modeling (MCM) environment incorporating these contributions. MCM seamlessly complements existing models at SAP. We show how service integration experts, architects, and testers can benefit from our approach that enables model-based integration testing and model verification facilities.

Wireless nodes equipped with multiple radio interfaces open up new fields of application. Ranging from multi-channel usage in a cell in order to increase the bandwidth to the creation of meshed multi-hop topologies. Using multiple wireless cards demands a large physical space, more energy consumption and as a consequence decreasing in the battery lifetime. Virtualization of the wireless network interface, which means to use a single wireless network interface to connect to more than one network simultaneously, seems to be a promising approach, since it allows us to realize the mentioned scenarios only with one radio interface. In this paper, we want to shed light on the state of the art and want to introduce new approaches to push this field beyond the current status.

Recently, several domain-specialized modeling languages have been developed to cope with the problems induced by the growing complexity of automotive embedded systems. For example, SysML, AADL, EAST ADL and AUTOSAR define high-level modeling languages that each propose certain forms of modularization and abstraction techniques to handle the system-level specifications of complex and heterogeneous systems. They thereby provide powerful support for the modeling of automobiles’ E/E systems architectures. But, in contrast to SDL that is based on communication diagrams and FSMs, these languages do not address the modeling of the behavior of the system under design, although this is a prerequisite for the partitioning, i.e. the definition of the architecture of the system. In order to state the guidelines for their enhancement, we discuss the shortcomings of these self-proclaimed domain-specialized modeling languages regarding the features that are required to support the partitioning and we present our solution to drive the partitioning of automobiles’ E/E systems specifications with the existing solutions.

Information systems used in health care use more and more a common approach of structuring and representing the health care data. One of the standards ad-dressing such an approach is HL7 Standard (Health Level Seven) [HL7 08]. It supports the inter-communication of various components in different medical facilities known as HIS (Healthcare Information Systems). So far, little attention has been given to a common methodology for assuring the quality of HIS systems and their testing. In this paper we present a test framework for HIS-based health care applications, which is based on the standardized test technology TTCN-3 providing both a testing language for specifying tests and a test architecture for the automated execution of tests. TTCN-3 has been successfully applied to test applications in telecommunication, automotive, etc., its application for health care systems is however in the beginning. We show how TTCN-3 can be applied to analyze medical information and how existent TTCN-3 based test methodologies can be re-used in the health care area. Due to the increased level of complexity of medical information used in HIS-based systems, manual testing is almost impossible and a very tedious task. Therefore, our framework aims at a higher degree of test automation.

This paper describes a benchmarking workload model for business process execution language (BPEL) engines for Web services. The proposed model is based on simulation of real world traffic conditions by defining a set of requirements which best characterize the end-users. The performance characteristics are evaluated on top of collected measurements such as success/fail rate, response times or round-trip delays which are then used to identify problems for scalability or usability aspects under heavy load.

The use of the standardised testing notation, testing and test control notation (TTCN-3) language has increased continuously over the last years. Many test suites of large sizes covering different domains exist. Therefore, it becomes important to provide the TTCN-3 community with methods and tools to evaluate the quality of tests. This paper presents the idea of evaluating the quality of the test data stimuli by using a data clustering method and measuring the coverage related to data clusters. A cluster contains stimuli which are considered similar for the system under test (SUT) behaviour; that means that each stimuli within a cluster should provide similar results from the test point of view.

Industry demands new approaches for test development to overcome the problem of increasing system complexity and efforts for testing. This contribution reflects practical testing conditions in different domains, the development steps for test derivation and emphasises the potentials for testing retrieved from an architectural system definition that has been enriched by pattern and contracts.

Similar to software development, the test development must be accompanied with a set of rules specifying how to write tests. They are grouped together into a document called guideline. Guidelines are especially necessary for large test specifications involving many developers and have the goal to reduce the effort of the overall development. So far, no universal guidelines for the TTCN-3 language [1] have been defined. Instead, each company or team defines and follows own development rules for test structuring and development. This paper deals with the problem of how to automate the validation whether a TTCN-3 test specification complies or not with an established guideline, i.e. guideline checking. The results of the validation process are a list of non-consistencies. A follow up step is the refactoring which automatically proposes and applies changes to improve the test suite compliance level, and thus its quality.

Test evaluation and test assessment is a time consuming and resource intensive process. More than ever this holds for testing complex systems that emanate continuous or hybrid behavior. In this article we introduce an approach that eases the specification of black box tests for hybrid or continuous systems by means of signal properties applied for evaluation. A signal property allows the characterization of individual signal shapes. It is determined by the examination of the signal’s value at time, the application of pre-processing functions (like first or higher order derivatives), and the analysis and detection of sequences of values that form certain shapes of a signal (e.g. local minima and maxima). Moreover we allow the combination of properties by logical connectives.The solution provided in this paper is based on terms and concepts defined for Continuous TTCN-3 (CTTCN-3) [12, 11], an extension of the standardized test specification language TTCN-3 [4]. Thus, we treat signals as streams and integrate the notion of signal properties with the notion of stream templates like they are already defined in CTTCN-3. Moreover, we provide a formal foundation for CTTCN-3 streams, for a selected set of signal properties and for their integration in CTTCN-3.

Over the past years, the scope of telecommunication services has increased dramatically making network infrastructure-related services a very competitive market. Additionally, the traditional telecoms are combined with Internet technologies for providing a larger range of services. The obvious outcome is the increase of the number of subscribers and services demand. Due to this complexity, the performance testing of continuously evolving telecommunication services has become a real challenge and requires efficient and more powerful testing solutions. This ability highly depends on the performance test design and on the efficient use of hardware resources for test execution.This paper proposes a performance testing methodology which copes with the characteristics mentioned above. The methodology consists of a set of methods and patterns, exemplified for the TTCN-3 language, to realize adequate performance tests for multi-service systems. The effectiveness of this methodology is demonstrated throughout a case study on IP Multimedia Subsystem (IMS) performance testing.

Embedded system and software testing requires sophisticated methods, which are nowadays frequently supported by application of test patterns. This eases the test development process and contributes to the reusability and maintainability of the test specification. However, it does not guarantee the proper level of quality and test coverage in different dimensions of the test specification. In this paper the quality of the test is investigated and numerous metrics are defined. They are based mainly on the applied test patterns. They give a measure of quality for the test design and executed test cases with respect to a number of aspects. They also evidence the value of patterns application. If weighted, they enable to assess the executed tests.

Reactive real-time software is used when safety is the issue and the margin for errors is narrow. Such software is used in automotive, avionics, air traffic control, space shuttle control, medical equipment, nuclear power stations, etc. As the timeliness of those systems is critical, it needs to be assured and tested. However, real-time properties require automated tests as manual tests are untimely and imprecise. This paper reviews Testing and Test Control Notation Version 3 (TTCN-3) as a means for real-time testing and proposes extensions to enable real-time test systems in TTCN-3. Small examples demonstrate the usage of the new constructs. Real-time operating systems are analyzed and reviewed, to enable the realization of real-time test systems based on TTCN-3.

Service-Qualität ist ein wichtiges Kriterium, das die Unterscheidung von Services gleicher Funktionalität ermöglicht. Wenn zukünftig vergleichbare Services verschiedener Anbieter miteinander konkurrieren, erlaubt der Faktor Service-Qualität eine Alleinstellung. Aber was ist eigentlich Service-Qualität?

This paper presents the idea of analyzing and refining the test data variance computation as a method to determine the quality of TTCN-3 tests. TTCN-3 as the only internationally standardized testing language is being used extensively by industry. A large amount of functional test specifications are written in this language. For this reason, a sensible issue is the quality analysis of these specifications. Data variance characterizes the test data distribution over the test system interface. TTCN-3 template distance constitutes the criterion to quantify the variance: ``similar'' and ``different enough'' test data. Due to the high flexibility of the language, the stimuli templates to be compared may not only contain concret evalues but also variables, function calls, parameters, values which are known at runtime only, etc. Therefore, we combine constraint programming (CP) with static analysis of TTCN-3 test suites targeting a realistic template solving. We name template constraint an expression whose value (or domain restrictions) can only be determined by looking at the execution paths in the analyzed test behavior. This method leads to a better computation of the data distance and, thus, to a better refinement of similarity classes. The approach is illustrated by an example which shows how data varies when the distance is computed taking into account the template constraints.

Software development and testing of enterprise resource planning (ERP) systems demands dedicated methods to tackle its special features. As manual testing is not able to systematically test ERP systems due to the involved complexity, an effective testing approach should be automated, also requiring that the appropriate test data has to be provided alongside. In this paper we identify four main challenges regarding the provision of test data for automatic testing of ERP software: system test data supply, system test data stability, input test data constraints and test data correlation. Several possible solutions to these challenges are discussed. We conclude with an outlook to possible research activities.

Despite the past few years’ intensive efforts of automobile manufacturers and their suppliers to enhance the quality assurance of their products, the problems of testing systems that, like automobiles, steadily increase in complexity and interconnectedness are still not solved. The variety of proprietary test systems and solutions do not allow an integrated definition, transfer, re-use and execution of tests for automobile manufacturers, suppliers and test equipment manufacturers. In contrast to the automotive industry, the telecommunication industry uses an integrated, manufacturer independent and standardized test technology: the Testing and Test Control Notation (TTCN-3). The advantages of this technology have already been recognized by the automotive industry and therefore, AUTOSAR decided to use TTCN-3 for the definition of functional tests. However, the components of automobiles do not only have functional requirements. In addition to functional input-output behavior, also continuous data streams and real-time behavior have to be tested. The current version of TTCN-3 has only limited capabilities for testing such non-functional properties. To overcome these limitations, a consortium from industry and academia started the research project TEst specification and test Methodology for Embedded systems in Automobiles (TEMEA). TEMEA is targeted on developing a TTCN-3-based test specification and test implementation methodology for embedded systems in automobiles. The project will develop solutions to test continuous data streams and real-time behavior with TTCN-3 and also investigate software engineering related issues like quality assessment and quality improvement of large test specifications. This presentation will introduce the TEMEA project and will present the first results of the initial requirements capturing phase of the project. It should stimulate discussions on future enhancements of the TTCN-3 language.

TTCN-3 has gained increasing significance in recent years. It was originally developed to fit the needs of testing software-based applications and systems in the telecommunication industry and has shown its applicability to a wide range of other industrial domains in the mean time. TTCN-3 provides platform-independent, universal and powerful concepts to describe tests, especially for discrete, interactive systems. However, TTCN-3 addresses systems with discrete input and output characteristics only. The lack of powerful means that reasonably allow specifying and evaluating continuous data flow makes TTCN-3 sufficient neither for the automotive industry nor for other industries that deal with highly complex software-based control systems. This paper introduces the notion of streams, stream ports and stream templates to TTCN-3. It revises the initial design of continuous TTCN-3, a TTCN-3 extension for testing continuous or hybrid systems [20,21] and demonstrates the applicability for a case study that is typical for testing embedded control systems in the automotive industry.

The design of the architectures of automobiles E/E (Electric/Electronic) systems consists in the allocation of the hardware platform and the distribution of the computing and the communication loads of the application software within the allocated hardware. This operation is called the partitioning. Following the actual model-driven design schemes, the input of the partitioning is generally a functional specification of the system under development in the form of communicating software components that must be mapped on the allocated hardware platform. However, even though these models are sufficient to describe the structure of a system, they are not good enough to support a CAD-supplied partitioning. They lack the facilities needed to support the analysis of the data flow and to investigate the closeness between the elements of the specification, thus to support the mapping. In this paper, we define the Components Data Flow Machine (CDFM), a modeling format that is defined to support the design of automobiles E/E systems architectures. The CDFMdefines the semantics of a synthesis model that results from a transformation of standard models like SysML, EAST ADL or AUTOSAR models.

The Testing and Test Control Notation (TTCN-3) is a widely established test technology traditionally used in the telecommunication domain. In its new version, TTCN-3 has a wider scope and applicability. It can be applied not only for testing the conformance and interoperability of communication protocols but also for testing the functionality, interoperation and performance of software-based systems in general. Therefore, TTCN-3 is nowadays used in other domains such as automotive, railways, avionics, or security systems. This chapter introduces the concepts of the TTCN-3 language and provides examples of its practical use.

Performance benchmarking is a method to assess performance characteristics of various systems across different chip/system architectures under realistic conditions, to collect measurements such as success/fail rate, response times or round-trip delays and to identify problems for scalability or usability aspects under heavy load. In this paper, a benchmarking workload model for IP multimedia subsystem (IMS) is introduced. Our performance methodology is based on simulation of real world traffic conditions by defining a set of requirements which best characterize a population of UEs (user equipment).

Quality models are needed to evaluate and set goals for the quality of a software product. The international ISO/IEC standard 9126 defines a general quality model for software products. Software is developed in different domains and the usage of the ISO/IEC quality model requires an instantiation for each concrete domain. One special domain is the development and maintenance of test specifications. Test specifications for testing, e.g. the Internet Protocol version 6 (IPv6) or the Session Initiation Protocol (SIP), reach sizes of more than 40.000 lines of test code. Such large test specifications require strict quality assurance. In this paper, we present an adaptation of the ISO/IEC 9126 quality model to test specifications and show its instantiation for test specifications written in the Testing and Test Control Notation (TTCN-3). Example measurements of the standardised SIP test suite demonstrate the applicability of our approach.

Auf der Grundlage eines ereignisbasierten Ansatzes zum Test von Web-basierten Systemen verfeinert dieser Beitrag die Beschreibung von Eingabedaten und modelliert charakteristische Merkmale von Eingaben anhand von Eingabeformularen. Eine Fallstudie validiert den vorgestellten Testansatz unter Berücksichtigung von ungültigen Eingabedaten.

A pool of competing modeling solutions have been proposed to cope with the problems induced by the growing complexity of automotive embedded systems, i.e. the E/E (Electric/Electronic) systems of automobiles. As the principal features of these solutions are axed around modularization and high level of abstraction, it is necessary to investigate their ability to support the implementation. An objective evaluation will be helpful to define how each modeling technique should be enhanced for a better support of the implementation, whenever necessary. This paper defines a framework to evaluate the capacity of modeling techniques to support the implementation in the automotive engineering domain, particularly the partitioning. Following the state-of-the-art in the partitioning of automotive embedded systems, we present the evaluation framework. Then, we introduce the most common modeling solutions used in the automotive embedded systems design and we use the framework to evaluate and classify them.

Functional testing of software dedicated for hybrid embedded systems should start at the early development phase and requires analysis of discrete and continuous signals, where timing constraints play a significant role. Test data generation is done manually nowadays, though it should be automated to the highest possible extent; system evaluation is mainly performed by application of reference signals, which however are not present during the design phase yet. Hereby, a concept for testing at the model level is introduced, due to which the transformation from requirements to test evaluation design occurs and finally semi-automatic derivation of test data is possible. There is no need for reference signals utilization, since characterization and extraction of particular signals' features are provided for the comparison during the final test. Furthermore, patterns of feature generators are built and templates are proposed.

Automating the generation of object-oriented unit tests is a challenging task. This is mainly due to the complexity and peculiarities that the principles of object-orientation imply. One of these principles is the encapsulation of class members which prevents non-public methods and attributes of the class under test from being freely accessed. This paper suggests an improvement of our automated search-based test generation approach which particularly addresses the test of non-public methods. We extend our objective functions by an additional component that accounts for encapsulation. Additionally, we propose a modification of the search space which increases the efficiency of the approach. The value of the improvement in terms of achieved code coverage is demonstrated by a case study with 7 real-world test objects. In contrast to other approaches which break encapsulation in order to test non-public methods, the tests generated by our approach inherently guarantee that class invariants are not violated. At the same time, refactorings of the encapsulated class members will not break the generated tests

Modellbasiertes Testen ist ein Schlagwort, unter dem verschiedenste Techniken zur Nutzung von Modellartefakten im Testprozess verstanden werden. Vor allem aber stellt der Begriff heraus, dass im Zuge der modellbasierten Systementwicklung entlang MDA und anderen Methoden ebenso modellbasierte Techniken zur Generierung, Gütebewertung, Ausführung und Wartung von Testartefakten genutzt werden sollten, um eine Qualitätssteigerung und Kostenreduktion beim Testen zu erreichen. Der Artikel stellt eine Klassifikation und Bewertung bestehender Methoden vor.

A major field in research and development are open, distributed, communication-intense systems, which are currently characterized by increasing heterogeneity in structure and technologies and by growing complexity and behavior dynamics. Autonomous systems are being investigated in order to return to manageable and stable systems whose operation and maintenance is affordable. By resolving the obstacles of current systems via enlarged system autonomy, the quality assurance of such dynamic, reconfigurable, adjustable systems becomes problematic. This keynote reviews requirements, methods and approaches for model-based quality assurance of autonomous systems.

Customizable mobile services are needed to address ever changing service requirements and shortened service development and deployment times. However, service flexibility is a natural evil for quality assurance: the service functionality and the overall system can no longer be guaranteed by the developer and tester. Instead, a post-customization test phase is needed, which is flexible enough to provide a good coverage of the customized service features. In order to make this post-customization testing efficient, reuse of rather general tests is proposed and outlined in this paper.

Test effectiveness is a central quality aspect of a test specification which reflects its ability to demonstrate system quality levels and to discover system faults. A well-known approach for its estimatation is to determine coverage metrics for the system code or system model. However, often these are not available as such but the system interface only, which basically define structural aspects of the stimuli and responses to the system. Therefore, this paper focuses on the idea of using test data variance analysis as another analytical approach to determine test quality. It presents a method for the quantitative evaluation of structural and semantical variance of test data. Test variance is defined as the test data distribution over the system interface data domain. It is expected that the more the test data varies, the better the system is tested by a given test suite. The paper instantiates this method for black-box test specifications written in TTCN-3 and the structural analysis of send templates. Distance metrics and similarity relations are used to determine the data variance.

Test suites for embedded systems are typically created from scratch using different, often inadequate methods. In consequence, industry branches dealing with software-intensive embedded systems have to cope with quality problems, even though test processes are particularly time-consuming and costly. Based on an evolving model-based testing methodology we introduce test design patterns for simplifying and accelerating the test specification process. Making use of patterns, complex test objectives are covered with lower effort and test cases contain fewer design errors.

Dieser Artikel beleuchtet technische Grundlagen sowie praxisrelevante Hintergründe, die 3GPP, WiMAX Forum und ETSI dazu veranlassten, im Rahmen ihrer Testaktivitäten TTCN-3 einzusetzen. Der Leser kann sich anhand des Artikels schnell einen Überblick über TTCN-3 basierte Testspezifikationen und Testwerkzeuge verschaffen.

Today’s telecommunication providers demand for flexible software solution with the goal to reduce the time to market for their marketing ideas significantly. The contribution of this paper is a new concept for testing the software adaptability. We present an approach which allows for testing the software with focusing on the coverage of customization possibilities. Additionally, we present the adaptation of the development process and tools which are necessary for the realization of the projects based on highly customizable service software.

TTCN-3 has gained increasing significance in recent years. Originally developed to fit the needs for testing software-based applications and systems in the telecommunication industry, TTCN-3 has shown its applicability to a wide range of other industrial domains in the mean time. TTCN-3 provides platform-independent, universal and powerful concepts to describe tests — especially for discrete, interactive systems — on different levels of abstraction. However, TTCN-3 addresses systems with discrete input and output characteristics only. In the automotive industry — as well as in other industries that deal with highly complex software-based control systems — this is not sufficient. Control systems often interact with their environment trough sensors and actuators using continuous signals. A test environment that adequately supports the specification, execution and evaluation of tests for embedded control systems has to provide concepts to handle this kind of signals. Moreover it has to support the test engineer with suitable abstractions that ease signal specification and signal evaluation.

Testing is an important step in the development of a software-intensive system, as it checks the compliance of a system to the (end) user requirements. In order to fulfil the market demands for better and faster system development, testing such systems has to be done in a more effective and efficient way. By more effective testing, a better indication of product quality is provided; by more efficient testing this indication is provided faster. The ITEA project TT-Medal has focused on the development of testing methods with advanced languages and the demonstration of these test methods in industrial case studies.

The aim of this paper is to present the means of black-box on-line test evaluation for hybrid real-time systems. The described procedures can be used for the model-based testing process so as to improve its effectiveness. In particular, intelligent automotive validation functions are considered, which are divided into different types depending on the nature of the evaluated issue. All provided definitions are specified on the meta-model level according to the Model Driven Architecture (MDA) trends. The application of the concepts given in this paper contributes to the continuous on-line test analysis and is exemplified by a component of an Adaptive Cruise Control model in Matlab/Simulink/Stateflowreg.

The systematic testing approaches developed within the telecommunication domain for conformance and interoperability testing of communication protocols have been extended and broadened to allow the testing of local and distributed, reactive and proactive systems in further domains such as Internet, IT, control systems in automotive, railways, avionics and alike. With the application of these testing principles it became apparent that the testing of systems with continuous systems is different to that of discrete systems. Although every continuous signal can be discretized by sampling methods (and hence mapped to the discrete signal paradigm), abstraction and performance issues in this setting become critical. This paper revises the initial design of Continuous TTCN-3. It presents the concepts for specifying continuous and hybrid test behavior. The TTCN-3 extensions are demonstrated for a case study.

Software reliability, an issue raised long time ago by software engineering researchers, is still a problem that must be faced nowadays. Our approach is to handle reliability of software products making use of a statistical distribution based model and historical data of similar projects. We analyze the software error metrics of large software projects developed in a multinational company in the last five years, searching for a Rayleigh pattern in the defect rate profile. If we can prove empirically such a pattern exists in certain environment, we have a rapid and powerful theoretical model to support project estimation decisions early in the development life cycle when historical data on similar projects is available.

The approach towards system engineering compliant to Model-Driven Architecture (MDA) implies an increased need for research on the automation of the model-based test generation. This applies especially to embedded real-time system development where safety critical requirements must be met by a system. The following paper presents a methodology to derive basic Simulink test models from Simulink system models so as to execute them in the same framework as the system model. The meta-models for Simulink and Test are explained and will enable automatic transformation in the future. The testing concepts of UML 2.0 Testing Profile (U2TP) have been partially adopted, however also enriched so as to deal with continuous functions, real-time constraints and to mirror Simulink specific features.

The design of TTCN-3 focused on extensions to address testing needs of modern telecom and datacom technologies and widen the applicability to many kinds of tests including performance tests. One of the most important features of TTCN-3 is the platform independence which allows testers to concentrate on the test specification while the complexity of the underlying platform (i.e., operating system, hardware configuration, etc.) is left behind the scene. As far as the test distribution is concerned, TTCN-3 provides the necessary language elements for distributed tests. This is however supported in a transparent fashion so that the same test may run either locally or distributed. The distributed execution of a test enables the execution of test components belonging to one test configuration on different computers (the test nodes), sharing thus a bigger amount of computational resources. Test distribution is a research challenge when it comes to the problem of how to distribute the test components efficiently on the test nodes. Specifically for load testing - a particular kind of performance test - we investigate strategies to distribute tests on heterogeneous hardware in order to use the hardware resources of the test nodes efficiently.

multi-firewall enterprise network is a challenge, especially if the auditing is meant to be a continuous process that should be seen as part of the operational functions of the network. Security auditing is the practice of evaluating the security of systems and networks. In this paper, we present a framework for automated network security auditing. The framework is limited to the problem of designing and deploying a network security auditing system that continuously verifies that the security policy enforcements points (i.e. firewalls) installed in an enterprise network continue to function asrequired. We point out the problems to be addressed in this proposed framework and present some solutions/algorithms to the problems. We also present our NetSecAuditor tool that we developed, following this proposed framework.

In the context of interoperability testing, TTCN-3 (Testing and Test Control Notation), already successfully used in testing middleware such as CORBA, CCM, EJB, matches perfectly in the field of Web services. This paper targets the development of TTCN-3 test suites forWeb services directly derived from WSDL interfaces. A mapping between WSDL data descriptions to TTCN-3 data is presented to enable the automated derivation of language artefacts and of basic test scenarios. These mapping rules and the the implementation support are evidenced by means of a case study.

The early integration of test development into the system development process becomes more and more important. By doing so, design mistakes and implemen-tation faults can be detected in an early stage of the system design and imple-mentation process, i.e. before the newly developed system is shipped to the cus-tomer. This allows for reducing the overall development time and costs significantly. This paper reports on results in integrating testing into a model-based development process. Based on a common MOF-based infrastructure, ini-tial test are automatically derived from the system models. Subsequently, an ex-ecutable test code can be automatically generated from the test models and be-ing applied to the final system for quality assessment.

Embedded systems play an ever increasing role for the realisation of complex control functions in many industrial domains - resulting in a big variety of requirements with respect to their dependability and reliability. Assuring dependability and reliability requires however different methods for hardware and software components. While the hardware reliability has to be guaranteed in particular for extreme operating conditions (temperature, mechanical influences, radiations etc.) and for possible material-dependent corrosion phenomena, the reliability of software components depend mostly on the correct implementation of the required functionality. Software-based control systems have specific characteristics, which - at least in their combination - are unique: they are typically embedded, interact with sensors and actuators with continuous signal characteristics, supervise discrete control flows, obtain and process simple and complex structured data, communicate over different bus systems and have to meet high safety and real time requirements. While different, also model-based development processes and methods for embedded systems exist, a generally recognized test technology for the analysis and evaluation of these systems, which lead to qualitatively high-quality, safe and reliable systems, is missing. Such a test technology has to address the different aspects of embedded systems and to enable the testing of discrete behaviours for communication and control sequences, of continuous behaviours for the regulation sequences and of hybrid behaviours for the control sequence in interaction with sensors/actuators and with other system components and the user. This paper presents a design of such a test technology for embedded systems, which is based on the standardized test technology TTCN-3.

Nowadays the use of complex distributed systems increases rapidly and the need for solid testing, too. Enterprise JavaBeans (EJB) is one of the most used standards for developing distributed systems. The applications based on EJB technology are secure, concurrent, transactional, etc. TTCN is in the telecommunication domain a widely established and used test technology. In its new version, TTCN-3 (Testing and Test Control Notation), it has a wider scope and applicability. It cannot be used only for testing the conformance and interoperability of communication protocols but also for testing the correct behaviour of distributed systems and application. This paper describes the language mapping from EJB/Java to TTCN-3 and an Eclipse based solution for testing EJB/Java based applications with TTCN-3.

An event-based approach for testing a web-based commercial system is presented. This approach is then compared with a state-based one along with a case study.

The systematic testing approaches developed within the telecommunication domain for conformance and interoperability testing of communication protocols have been extended and broadened to allow the testing of local and distributed, reactive and proactive systems in further domains such as Internet, IT, control systems in automotive, railways, avionics and alike. With the application of these testing principles it became apparent that the testing of systems with continuous systems is different to that of discrete systems. Although every continuous signal can be discretized by sampling methods (and hence mapped to the discrete signal paradigm), abstraction and performance issues in this setting become critical. This paper investigates the different options to support tests of embedded control systems in the framework of TTCN-3.

TT-Medal is a European research project on tests and testing methodologies for advanced languages. In TT-Medal key roles are assigned to international standards, the Testing and Test Control Notation (TTCN-3) by ETSI and ITU-T, the Unified Modelling Language (UML2.0) and its testing profile by the OMG. Reading this white paper provides you a summary on the state of the art of test tools, an introduction to TT-Medal achievements, in particular insight into the industrial case studies that have been performed in the project. Three major directions for research are addressed: a common test tool infrastructure, approaches for the automatic generation of TTCN-3 tests, and the integrated system and test development. Our conclusion picks up these directions and attempts to describe a future picture for automated testing in the next years.

Self-monitoring is one of the key expected capabilities of an autonomic system. An autonomic system can be a single network node or the entire network as an entity, having the ability to automatically adjust its behaviour based on the conditions in which the system and its components work. Self-monitoring for the purposes of self-configuration, cooperative event detection by a number of systems in a network, knowledge/information distribution, service-diagnosis and self-protection requires a number of challenges/questions to be addressed. In this paper, we present the concepts behind self-monitoring as a capability of a single node and as a capability of a network that is considered as an entity. We also discuss the challenges and questions to be addressed when designing and deploying self-monitoring mechanisms for a single node and for an entire network and, we present some solutions to these challenges. Because self-monitoring is broad, we limit our focus to self-monitoring applied to inbound/outbound protocol-specific traffic at some point(s). As part of the solution to some challenges we try to address, we introduce the concept of on-demand monitoring (ODM) of protocol-specific traffic.

System-level testing considers functionality and load aspects to check how a system performs for single service requests and scales as the number of service requests accessing/using it increases. This paper presents a flexible test framework including functional, service interaction and load tests. It is generic in terms of being largely independent of the system to be tested. The paper discusses the automation of the test framework with the Testing and Test Control Notation TTCN-3 and also presents an implementation of the test framework using a TTCN-3 toolset. The test framework is exemplified for Web service tests and demonstrates distributed functional and load tests for a specific Web service.

The approach towards system engineering according to Model-Driven Architectures (MDA) with code generation derived from model implies also an increased need for research on automation of the test generation process. This paper presents an approach to derive executable tests from UML 2.0 Testing Profile diagrams automatically. In particular, an approach to derive executable tests within the Testing and Test Control Notation (TTCN-3) is discussed. The transformation rules between the source U2TP meta-model to the target TTCN-3 meta-model are given.

Assembling new software systems from prefabricated components as an attractive alternative to traditional software development practices is more and more investigated. Component technologies like CCM. Net or EJB are accompanied by model-based approaches like MDA. However, still the emphasis is rather on system design and development and not on system validation and testing. The expected reductions in development time and effort will only arise if separately developed components can be made to work effectively together with minimal effort. Lengthy and costly in-situ verification and acceptance testing directly undermines the benefits of heterogeneous components and late system integration. This paper extends contract-based built-in tests where components are equipped with the ability to check their execution environment at run-time with approaches to derive built-in tests from system models, represent them on model level, and to generate executable tests from these test models. This model-based approach increases the automation level in generating and realizing built-in tests and therefore also increases the quality of and reduces needed resources for developing built-in tests.

SummaryBuilt-in contract testing is based on the idea of building tests directly into components, so that each component can assess whether it will be integrated into a suitable environment, and the environment can assess whether a newly integrated component will be acceptable.Component technologies such as CCM, .NET, or EJB are more and more being supported by model-based approaches like the OMG’s Model Driven Architecture (MDA). The idea is to gain considerable momentum in the development of component-based architectures through high-level modeling and automatic code generation. However, the emphasis with these approaches is currently more on system design and development and not as much on system validation and testing, so the expected reductions in component-based development in terms of time and effort can only be realized in component and application construction and not during test development. Lengthy and costly in-situ verification and acceptance testing, that is mainly still performed manually, undermines the benefits of these modern development approaches.This chapter demonstrates how built-in contract testing can be integrated with and made to supplement automatic approaches to derive application testing from system models, represent them on the model level, and generate executable tests from these models. This model-based testing approach increases the degree of automation in generating and realizing built-in contract tests; therefore, it also increases the quality of the built-in tests and reduces the resources required for developing them.

TTCN (Testing and Test Control Notation) is in the telecommunication domain a widely established and used test technology. In its new version, TTCN-3 has a wider scope and applicability. It cannot be used only for testing the conformance and interoperability of communication protocols but also the interaction of e.g. sensors, actuators and control units connected via bus systems. Therefore, TTCN-3 is now being used in other domains such as automotive, railways, avionics and security systems. This paper gives an overview on TTCN-3, describes basic concepts and presents results on the application of TTCN-3 for testing MOST based components.

Patterns are used in various engineering disciplines to represent common aspects of a set of solutions to a (set of) common problem(s). This paper discusses main concepts of test patterns, provides a characterization of test patterns and describes the use of test patterns in the test development process. Test patterns can be used to support the vertical reuse between different testing phases and testing kinds, horizontal reuse between different product version and historical reuse between different product versions. Specification means for test patterns will be analysed and compared with the language features of the Testing and Test Control Notation (TTCN-3) [8].

Abstract: Model centric development and engineering according to Model-Driven Architectures (MDA) has recently gained much attention. This paper presents an approach how a model centric approach cannot only be used for system development but also at the same time to support the provision of system tests which are to be an integral part of the overall system development. The paper discusses the various artefacts for model-based testing along MDA and their relation to the artefacts in system development. The testing artefacts can be designed and modelled with the UML 2.0 Testing Profile (U2TP) which extends UML 2.0 with test specific concepts. 1

This contribution presents a generic approach and implementation for protocol detection and traffic analysis (TA). Self-detection and auto-configuration facilities minimize the effort to manually configure a monitor or test device and should be an integral part for test automation. We elaborate our concept on the intelligent self-detecting features for a MTS monitor focussing on interfaces, protocols and signaling scenarios in the context of the UMTS Terrestrial Radio Access Network (UTRAN).

The Testing and Test Control Notation TTCN-3 is a test specification and implementation language that has been defined classically in form of a formal syntax and a semiformal semantics. UML, MOF and MDA has shown that on contrary meta-model based language definitions impose new ways of defining language semantics as a separation of syntax and semantic concept space, of integrating languages via a common meta-model base and of generating and deriving models from other models via model transformers. This paper defines a meta-model for TTCN-3, such that TTCN-3 can take advantage of meta-model based approaches – in particular for integrating test but also system development techniques. It also discusses the realization of the TTCN-3 meta-model and its use for meta-model based tools.

Security issue is an increasing concern and pressures are increasing on organizations to take a more systematic approach to incorporating security into their systems. An important component of security requirements is access control. In this paper, we introduce a variant of the role based access control model that assigns role to both partners of an interaction. We further develop a methodology to determine whether a set of access control policies is contradictionary, i. e, contains a conflict. Model checking techniques are employed for conflict analysis.

Traditionally the testing and test control notation TTCN has been used and accepted widely in the telecom industry. The new version TTCN-3 is intended to be applicable to a broader scope of communication and software systems. This paper describes the initial results of a research project with the aim of evaluating the use of TTCN-3 within the automative telematics domain. The paper addresses the use of TTCN-3 to specify the reusable test cases for the telematics domain, the development of a TTCN-3 test platform for telematics systems and future work required before the requirements of the industrial testing processes within the automotive industry can be met.

In this paper we will present, how a model centric approach cannot only be used to rapidly develop the system but also at the same time to support the provision of the system tests which is an integral part of the overall system development. The key technology used to achieve this is a set of meta-tools which contains model repository generators and model transformer generators.

Im September 2000 wurde TTCN-3 durch ETSI - dem Europäischen Telekommunikationsstandardisierungs-institut - erstmals der öffentlichkeit vorgestellt. Seither hat sich TTCN-3 sowohl im Anwendungs- als auch im Werkzeugbereich etabliert. Eine Vielzahl von Nutzern setzen TTCN-3 für die Entwicklung von Testlösungen für funktionale, Konformitäts-, Interoperabilitäts-, Skalierungs-, Lasttests, etc. für Protokolle, Dienste, objekt- und komponentenbasierte Systeme, für SmartCards, im Automobilbereich, etc. ein. Eine Palette von Werkzeugen verschiedener Anbieter steht dem Nutzer zur Verfügung. Eine Reihe von standardisierten Test Suiten in TTCN-3 wie für SIP, SCTP und IPv6 oder aber Test Frameworks wie für CORBA basierte Systeme und Web Services können unmittelbar genutzt werden. Doch auch der Standard hat sich weiterentwickelt. Nach der Vorstellung der ersten Version von TTCN-3 (im Wesentlichen bestehend aus der Core Language und dem tabellarischen Präsentationsformat) sind als weitere Bestandteile die operationale Semantik, das graphische Präsentationsformat (GFT), die Ausführungsschnittstellen zur Anbindung an das zu testende System (TRI) und zur Anbindung an die Testplattform (TCI) als auch Einbettungen externer Typsysteme hinzugekommen. Damit steht mittlerweile ein stabiler und umfassender Ansatz zur abstrakten Testspezifikation- und -implementierung zur Verfügung. Ein kontinuierlicher Wartungsprozess für TTCN-3 bei ETSI ermöglicht die Korrektur, Verbesserung und den weiteren Ausbau von TTCN-3 entsprechend der Nutzererfordernisse. In diesem Beitrag wird auf wesentliche Elemente und Konzepte von TTCN-3 eingegangen. Darüber hinaus wird kurz auf änderungen in TTCN-3 eingegangen, die in die aktuelle Version eingeflossen bzw. für die nächste Version in Planung sind.

TTCN-3 allows an easy and efficient description of complex distributed test behaviour in terms of sequences, alternatives, loops and parallel stimuli and responses. The test system can use any number of test components to perform test procedures in parallel. Features which are not directly supported in TTCN-3 can be included by the use of external types, data and functions. However, the access to external test behaviour is rather limited as external functions are executed as plain functions only without being performed on test components. The combination of external functions and test components allows the extension of the communication between the TTCN-3 test system and the external behaviours towards complex send/receive patterns and supports the distribution of external behaviours on remote nodes. This paper discusses the concepts of external behaviours, proposes extensions to TTCN-3 to realise them and demonstrates its application by a concrete example.

This paper discusses the results and experiences gained from the project Model Driven Development of Telecommunication Systems (MDTS) which is a joint project of three institutes of the Fraunhofer Society. The main goal of the project was to define a model driven software construction process, which enables the integration of system development and system test. It included the provision of a set of supporting tools to increase the level of automation in this process. The developed concepts have been applied in the telecommunication domain using an example application. This application supports mobile workers in critical industrial domains like aircraft maintenance or chemical production.

Testing often accounts for more than 50% of the required effort during system development. However, testing is often not well integrated with other development phases. One reason for this is that designers, developers and testers all use different languages and tools, making it difficult to communicate with each other and to exchange documents. The UML 2.0 Testing Profile bridges the gap between designers and testers by providing a means to use UML for test specification and modeling. This allows the reuse of UML design documents for testing and enables test development in an early system development phase. The testing profile provides support for UML based model-driven testing. This paper presents the concepts defined in the UML 2.0 Testing Profile and explains their usage by applying those to an example of a simplified Automated Teller Machine (ATM).

The UML 2.0 Testing Profile (U2TP) is the first profile based on UML 2.0. Although UML 2.0 itself provides means for time specifications, U2TP added new time concepts for testing specific needs. The motivation for the U2TP time concepts come from different languages and profiles with real-time features, including the UML Profile for Schedulability, Performance and Time Specification (SPT). This paper provides an overview about time concepts, primarily concentrating on the UML 2.0 and U2TP time concepts. Since time concepts are not only important for test specifications, the U2TP time concepts may be considered for adoption in a UML 2.0 real-time profile.

The Session Initiation Protocol (SIP) has been continuously gaining acceptance over the last years, as a valuable alternative to ITU-T’s H323 in providing call signalling capabilities for audio, video and other multimedia communication over Internet Protocol (IP) based networks. With the number of SIP implementations growing at an extremely fast rate, the need for testing suites with regard to conformance, interoperability and QoS is becoming more and more crucial. While interoperability and conformance have been discussed and tested by several related work and test suites, the performance aspects of SIP implementations have (amazingly) been granted fairly low attention. We present in this analysis an architecture for evaluating the signalling performance of SIP implementations.

Management of services over large distributed systems together with the growing complexity of underlying technologies has become a challenge in the present time. Faults can occur anywhere in a distributed system. Online validation addresses the activity of evaluating the behavior of services and service environments in the production phase in order to determine functional faults immediately. In this paper, we present our work on the design and implementation of an online validation platform. We describe a generic architecture, which can be instantiated for implementing validation systems for any type of distributed environments. We also present our experiences in using TTCN-3 to implement an online validation system for active network environments.

The ongoing challenge of supporting high performance communications, real-time critical and secure applications leads to a shift from traditional IP networks towards a more intelligent, end-to-end, service-aware network paradigm. Future networks have to i) be more content or application-aware; ii) provide dynamic features for service creation; iii) observe and enforce network-wide policies; and finally, iv) enable control from the network provider to the end-user. This paper presents the technical aspects of a policy based auditing system for a QoS-enabled network. The goal is to define the architecture of policy-based network QoS control and to develop and implement a prototype to evaluate this architecture.

The testing and test control notation (TTCN-3) is a new test specification and test implementation language that supports all kinds of black-box testing of distributed systems. TTCN-3 was developed in the years 1999 to 2002 at the European Telecommunications Standards Institute (ETSI), as a redesign of the tree and tabular combined notation (TTCN) standard (ITU-T Rec. X.292). TTCN-3 is built from a textual core language that provides interfaces to different data description languages and the possibility of different presentation formats. This makes TTCN-3 quite universal and application independent. TTCN-3 is being published as the ITU-T Rec. Z.140 series. This paper provides an introduction to TTCN-3. This includes an overall view of the TTCN-3 core language, a description of the existing presentation formats, an explanation of the implementation of TTCN-3-based test systems and a discussion about the current usage and the future of the language. The authors all participated in the work within ETSI.

Web services provide seamless connections from one software application to another over private intranets and the Internet. The major communication protocol used is SOAP, which in most cases is XML over HTTP. The exchanged data follow precise format rules in the form of XML Document Type Definitions or more recently the proposed XML Schemas. Web service testing considers functionality and load aspects to check how a Web service performs for single clients and scales as the number of clients accessing it increases. This paper discusses the automated testing of Web services by use of the Testing and Test Control Notation TTCN-3. A mapping between XML data descriptions to TTCN-3 data is presented to enable the automated derivation of test data. This is the basis for functional and load tests of XML interfaces in TTCN-3. The paper describes the mapping rules and prototypical tools for the development and execution of TTCN-3 tests for XML/SOAP based Web services.

Obwohl Testen oftmals bis zu 50% des gesamten Entwicklungsaufwands erfordert, ist es bislang nicht bzw. wenig in die einzelnen Systementwicklungsphasen integriert. Das liegt u.a. daran, dass Architekten, Entwickler und Tester verschiedene Sprachen und Werkzeuge nutzen, so dass die Interaktion untereinander und der Austausch von Informationen und Dokumenten erschwert werden. Das UML 2.0 Testing Profile (U2TP [1][2]) schließt diese Lücke, indem Konzepte und Wege aufgezeigt werden, wie UML auch für den Testentwurf und Testspezifikationen eingesetzt werden kann. Das ermöglicht die Wiederverwendung von Systemdokumenten für die Testentwicklung und erlaubt so eine frühzeitige und kontinuierliche Verzahnung von Systementwicklungs- und Testschritten. U2TP ist die Grundlage für modell-basiertes Testen im Kontext von UML.

Distributed test setups for efficient load, performance, scalability, interworking, and end-to-end tests are gaining importance for the assessment of distributed communicating systems. The Testing and Test Control Notation TTCN-3 provides concepts for component-based distributed test systems in dynamic test configurations, where test components may reside on various network nodes to be near the interfaces of the tested system. The realization of executable TTCN-3 tests on concrete test platforms involves TTCN-3 compilation/interpretation and adaptations to the test platform. The TTCN-3 Control Interfaces TCI define entities, interfaces, types and operations needed to flexibly manage and distribute TTCN-3 based test systems. It complements and completes the TTCN-3 Runtime Interface TRI. This paper discusses the underlying concepts of TCI and demonstrates its use for the realization of a distributed test for the Session Initiation Protocol SIP.

System-level testing considers functionality and load aspects to check how a system performs for single service requests and scales as the number of service requests accessing/using it increases. This paper presents a flexible test framework including functional, service interaction and load tests. It is generic in terms of being to a large extend independent of the system to be tested. The paper discusses the automation of the test framework with the Testing and Test Control Notation TTCN-3. The test framework is exemplified for Web service tests.

CORBA-basierte Systeme werden im TK und IT Sektor zur Realisierung verteilter Anwendungen wie im E-Business oder für Internet-Lösungen genutzt. Mit dem zunehmenden Einsatz CORBA-basierter Systeme in geschäfts- und sicherheitskritischen Bereichen nimmt der Bedarf an zuverlässigen Testlösungen zu. Mit TTCN-3 – der Testing and Test Control Notation – und der Anbindung von TTCN-3 an IDL – der Beschreibungssprache für Schnittstellen CORBA-basierter Systeme – steht nun eine Testlösung für systematische Tests auf Funktionalität, Leistungsfähigkeit, Sicherheit, etc. von CORBA-basierten Systemen zur Verfügung.

Graphical system design techniques like Message Sequence Chart (MSC) and Unified Modelling Language (UML) are gaining more and more acceptance because they ease the development, understanding, and maintenance of software systems. In the testing area no accepted graphical test specification and implementation techniques exist. To overcome this shortcoming, a graphical presentation format for the Testing and Test Control Notation (GFT) has been defined. GFT supports the graphical design, implementation, visualization, documentation and tracing of test behaviour. GFT is based on MSC and extends it with test specific concepts like verdicts and defaults. GFT is also the basis for the definition of a UML testing profile to enable the integrated system and test development with UML models. This paper discusses GFT and its relation to MSC and to the UML testing profile.

UML models focus primarily on the definition of system structure and behaviour, but provide only limited means for describing test objectives and test procedures. However, with the approach towards system engineering with automated code generation, the need for solid conformance testing has increased. In June 2001, an OMG Request For Proposal (RFP) on an UML2.0 Testing Profile (UTP) has been initiated. This RFP solicits proposals for a UML2.0 profile, which enables the specification of tests for structural and behavioural aspects of computational UML models, and which is capable to interoperate with existing test technologies for black box testing. This paper discusses different approaches for testing with UML and discusses the ongoing work of the Testing Profile. Special emphasize is laid on the mapping of UML2.0 testing concepts to the standardized Testing and Test Control Notation (TTCN-3).

This paper presents the concept of test frameworks and the specification and implementation of test frameworks in TTCN-3 being the Testing and Test Control Notation for black-box and grey-box tests of reactive systems. Test frameworks enable an easy reuse, extension and customization of predefined test patterns for test data, configuration, behavior and control. Test patterns can be combined and customized to yield additional tests for further system features, changed system characteristics as well as for additional test requirements. The implementation of test frameworks and the execution of resulting tests are supported by a TTCN-3 tool chain providing means for graphical test specification, compilation, management and execution. A GPRS example demonstrates the practical application of test frameworks using a set of TTCN-3-based tools.

Deploying a distributed application on the nodes of a network has undoubtedly been a daunting task, which encompasses the distribution as well as the configuration of its components. Deployment and configuration concerns the placement of software components onto the nodes of a target environment, their installation, configuration, and instantiation, as well as the distribution and configuration tasks during the service usage phase. However, they are not fully addressed by the current CORBA component model (CCM). This paper elaborates on the platform architecture for deployment support and proposes the necessary extensions to current CCM.

One of the most provocative research areas in software engineering field is the testing of modern component based distributed applications in order to assure required quality parameters. Dynamic interactions and structural embedding, run-time loadable configurations, and services that can be deployed in arbitrary executions environments results in an increased complexity. Moreover, that the variety of possible states and behaviors becomes unpredictable. Thus, since testing during the development phase is always applied in simulated environments, it is almost impossible to detect faults, which appear under real condition, during production phase of a system. We therefore aim at concepts and methodologies that achieve on-line testing of distributed component based systems in their production phase. In comparison with off-line testing (i.e. testing that takes place during system development), on-line testing addresses particular aspects of the behavior of distributed systems, such as: functionality under limited time and resources available, complex transactions that are performed between components provided by different vendors, deployment, and composition of different services.

Recently, international standard bodies approved the third evolution of the TTCN test specification language (TTCN-3) as a requirement to modernise and widen its application beyond pure OSI conformance testing. Even though TTCN-3 makes the description of complex distributed test behaviour much easier there is still a requirement from the user community to provide a visualisation means for test specification, development and tracing. Message Sequence Charts (MSC) appeared to be a particularly attractive candidate as a graphical means for visualising TTCN-3 test cases.

This paper presents a new approach to interoperability testing of CORBA implementations. The ORB Under Test (OUT) is embedded in the test system in such a way, that the behavior of either the client side or the server side of the OUT is controllable and observable. Components of the test system emulate the peer entities of the OUT and the applications based on the OUT in order to get direct access both to the GIOP interface and to the CORBA APIs. A specification framework for the test system is proposed. It separates the concerns of test data and test behavior to facilitate modularity and dynamic of the test system. The description languages CORBA IDL, ODMG ODL and TTCN-3 are used in the framework

Middleware platforms are in widespread use for distributed systems. Their quality is key to the stability and interoperabilily in multi-vendor heterogeneous environments. It is the aim of the EC IST project CORVAL2 to enhance the techniques used to validate the conformance of OMG's CORBA technology. The paper investigates testability aspects of CORBA ORBs and considers CORBA based systems both from a theoretical and practical view on testing. Test strategies are proposed and a conformance test suite presented

Recently, the European Telecommunications Standards Institute (ETSI) approved the third edition of the Tree and Tabular Combined Notation (TTCN-3) as a requirement to modernise and widen its application beyond pure OSI conformance testing. As part of this evolution, TTCN is embracing Message Sequence Charts (MSCs) as a natural notation for specifying and visualising test suites. This paper defines the role of MSCs during test development, and more specifically introduces an MSC profile called the Graphical Format for TTCN (GFT) that facilitates the effective specification of TTCN-3 test suites.

This paper presents our findings with the Testing and Test Control Notation (TTCN-3). Using recently developed tools, we have successfully implemented a TTCN-3 test suite for the GIOP/IIOP standardized CORBA interoperability protocol. Of particular interest are TTCN-3 improvements over its predecessor Tree and Tabular Combined Notation (TTCN-2). This paper described the GIOP test suite, its design and implementation and compares it to a TTCN-2 test suite for GIOP. With our comparison, we specifically address test developers with technical knowledge on at least one of the TTCN versions.

Tests for Operation-based Interfaces can be implemented with concrete programming languages like C++ or Java. To have test system independent test descriptions and to gain better test reusability, in many companies more abstract test specifications are applied in this context. This paper discusses our experiences from a test suite development project for the OMG CORBA specification. In particular, the applicability of two major techniques- ADL and TTCN- has been investigated. This paper reports on the experiences and gives engineers some help to decide between the approaches for test development for operation-based interfaces.

The European Research Consortium for Informatics and Mathematics (ERCIM) has recently celebrated its 10th anniversary. The ERCIM Working Group on Formal Methods for Industrial Critical Systems (FMICS) is organizing its 5th International Workshop. FMICS workshops are dedicated to interested researchers at ERCIM sites, universities and industry active in the industrial application of formal methods. Among a variety of formal methods conferences and workshops FMICS is increasing its popularity. The idea of FMICS workshops is to attract people with industrial relevant topics, with internationally well-known invited speakers and with high-quality technical papers in combination with a discussion podium for the exchange of ideas. The workshop character of FMICS is realized on a minimal cost base. This time, FMICS is organized right after ETAPS'2000 - the European Joint Conferences on Theory and Practice of Software in Berlin. After starting the FMICS workshop series 1996 in Oxford (UK) further workshops followed 1997 in Cesena (I), 1998 in Amsterdam (NL) and 1999 in Trento (I). In 2000, the workshop is hosted and organized at the GMD Research Institute for Open Communication Systems (FOKUS) in Berlin, Germany. This year' workshop includes sessions on modelling, verification, testing and software development, MSC/SDL, and various applications and case studies. We are pleased to present two interesting invited talks: Günter Karjoth, IBM Zurich (CH), addresses the value of formal methods for security properties such as confidentiality and authenticity. Holger Hermanns, University of Twente (NL), investigates in the performance and reliability model checking and construction. We wish to thank the members of the programme committee, especially the FMICS working group chairman Hubert Garavel, for the excellent assistance during the planing of the workshop, the invited speakers, the authors and the reviewers for their scientific contributions, the people from the GMD Fokus Competence Center TIP for preparing the workshop event, and ERCIM and GMD for their financial and organizational support of FMICS.

Recently, the European Telecommunications Standards Institute (ETSI) approved the third edition of the Tree and Tabular Combined Notation (TTCN-3) as a requirement to modernized and widen its application beyond pure OSI conformance testing. TTCN-3 is a text-based language for the specification of tests for reactive systems. In addition to several new concepts for defining test behavior, it supports besides textual test specifications a graphical forma as a natural and user-friendly notation for developing, documenting and visualizing test suites. This paper introduces the main concepts of TTCN-3, presents its graphical format based on Message Sequence Charts (MSC) and shows the application of TTCN-3 for testing text-based protocols - a new area of application for systematic testing.

The Tree and Tabular Combined Notation (TTCN) is a semi-formal notation which supports the specification of abstract test suites for protocol conformance testing. An abstract test suite is a collection of abstract test cases1). As indicated by the name TTCN, test cases are described in the form of behavior trees and different kinds of tables are used for the graphical representation of test suites.

The development of an Message Sequence Chart (MSC) based graphical representation of the Tree and Tabular Combined Notation (TTCN) is part of the ETSI project STF 156 on 'Specification of a Message Sequence Chart/UML format, including validation for TTCN-3'. Experiments with different kinds of MSC representations show that certain extensions of the MSC language are requested in order to obtain a sufficiently transparent and readable MSC test format. Extended HMSCs where reference symbols may either contain hypertext–like descriptions or, in the expanded form, the detailed event structure of basic MSCs appear to be especially suitable for a compact and transparent test case representation. For an effective usage of such advanced MSC constructs, a corresponding tool support appears to be mandatory where the event structure of special paths in the test case is described explicitly while others are hidden in MSC references containing pure textual descriptions. We propose the name 'HyperMSCs' for such extended HMSCs. A significant difference between the standard TTCN format and the MSC format refers to the behavior description of several test components. In the concurrent case, a TTCN test case describes the communication of the main test component (MTC) and its ports. The behavior description of parallel test components (PTCs) is provided by separate functions. A simple translation of this partitioning into MSCs leads to separate MSC specifications for the different test components which have to be merged somehow by an appropriate join operation. However, such a ‘local’ view does not make use of the main advantage of the MSC language. The strength of MSC lies in the immediate description of the communication behavior which is provided by a ‘global’ view including all test components in the same diagram. The translation of TTCN into the MSC format is demonstrated using a simple example taken from the Inres protocol.

The paper introduces a new concept to express the architecture and behavior of distributed systems in a formal, detailed and extensible manner in terms of reference point facets (RP-facets). RP-facets are based on the well-established concept of reference points as used in ODP and TINA. Facets describe statical and dynamic aspects of reference points as well as pre- and post-conditions for their use. The paper gives a mathematical characterization of RP-facets, defines a specification template for the definition of RP-facets and derives a conformance test method for their validation. An example taken from the TINA retailer reference point shows the application and practical use of RP-facets.

The development of an MSC (Message Sequence Chart) based graphical representation of TTCN (Tree and Tabular Combined Notation) is part of the ETSI project STF 156 on 'Specification of a Message Sequence Chart/UML format, including validation for TTCN-3'. Important language constructs of TTCN can be translated into corresponding MSC constructs in a straightforward manner. However, it turns out that without certain extensions and modifications, the ITU standard language MSC is not capable to produce a sufficiently transparent and readable graphical representation of TTCN. In order to arrive at a really convincing solution, High-level MSCs are re-interpreted in a way, which has an analogy in hypertext-like specifications. MSC references may be shown also in an expanded manner and non-expanded MSC references may contain hypertext-like descriptions instead of pure reference names. As a result, there does not exist a strict borderline between High-level MSCs and Basic MSCs any longer. Such a generalisation of MSC is really efficient only together with a corresponding tool support, which allows the smooth transition between different levels of detailed description similarly to hypertext. Therefore, the name 'HyperMSC' is proposed for such extended High-level MSCs. The paper demonstrates how HyperMSCs arise from standard MSC descriptions in an evolutionary process of stepwise improvement and simplification.

RM-ODP (Reference Model of Open Distributed Processing) is a reference model for open information processing systems. Conformance is one of the key aspects of open systems. Conformance ensures the interworking in a multi-vendor information processing environment that may evolve over the time. An essential partitioning concept for open system is that of reference points (RPs). RPs consist of a set of interfaces together with potential interactions at these interfaces. RP specifications define conformance requirements, so that they can be used to determine the conformance of a system. However, the testing of RPs of real systems is restricted as their functionality is often very complex. Refinement of the RP structure is required to ease both their realization and also their testability. In particular, the incremental development of RPs that allows both the evolution of the system architecture and the step-by-step implementation by vendors should be supported. A new concept — that of an RP-facet — has been recently defined to structure RPs and to improve their testability. The focus of this paper is on structuring principles and dependence relations used by the RP-facet concept. They serve as a basis for incremental and efficient testing at RPs. An example taken from use scenarios on service access in open systems is elaborated in the paper.

TINA combines modern methodologies and techniques, such RM-ODP [11] and CORBA [15] to support the development of large-scale telecommunication systems. TINA provides an open architecture for a multi-vendor environment. The TINA architecture addresses a wide range of issues and provides a complex set of concepts and principles. It has been partitioned into several models, subsystems, components, etc. in order to handle the complexity. An essential partitioning concept is that of reference points (RPs). Reference points consist of a set of interfaces together with potential interactions at these interfaces. Reference point specifications define conformance requirements for a relationship between or within administrative domains of distributed systems. The TINA reference point concept follows the RM-ODP conformance assessment principles [11]. An example for an inter-domain reference point is the Retailer Reference Point [23]. Key issues for multi-vendor systems is interoperability. Reference points as a collection of conformance requirements are the basis to increase the likelihood for interoperability. Conformance testing is an effective and efficient means to validate the overall functionality of a multi-vendor system. It is a wellestablished alternative to the otherwise needed many-to-many test setups for individual components and/or sub-systems to ensure their interoperability.

Modern distributed systems are component based. Due to the distributed nature and complexity of such systems, new requirements concerning their testing arise. Components communicate via well-defined interfaces. The target of testing is to validate the behavior of (a set of) components at their interfaces. The paper presents an approach to build a test system itself from test components, which interact with the components under test via their interfaces. For the development of such test systems, a framework is proposed which contains base types for test components with generic behaviour for the setup and configuration of a test system and for the exchange of coordination messages between test components. For the specification of a test system the same method as for the specification of the system under test is proposed: UML. The paper presents generic test components of the framework and how concrete test components can be derived. The test framework can be used not only for the development of functional tests but also for e.g. performance and load tests. An example of using the test framework is given.

Component-based technologies are recently used to implement complex distributed systems. Testing of these systems becomes even more complicated. New test methods need to be investigated. Components of a system under test communicate via well-defined interfaces. It is to validate the behavior of (a set of) components at their interfaces. The paper proposes to build a test system itself from test components, which interact with the components under test via their interfaces. For the development of such test systems, a UML framework is used which contains base types for test components with generic behavior for the setup and configuration of a test system and for the exchange of coordination messages between test components. The paper discusses the aspect of automatic code generation of test components from UML specifications of test systems. An example of using the test framework is given.

This paper presents means to test the functionality, scalability and performance of distributed telecommunication applications based on CORBA ORBs. The presented performance test approach is generic and can be used to cope with various test objectives. For example, it can be used to evaluate the maximal performance of a server device in a networked environment. It is also of use to evaluate response times of servers and the supported number of simultaneous clients. The test manager Ttman presented here is a tool to setup and parameterize distributed test configuration and to control the test execution. Exemplarily, a TINA access session which is part of the TINA platform developed by GMD FOKUS is considered.

The Reference Model for Open Distributed Processing (RM-ODP) describes an architecture for the design of distributed services where the basic idea is to split the design concerns into several viewpoints. This is in order to overcome the immense complexity of today’s distributed systems by structuring the design process. This paper presents an integrated approach not only covering the field of service design but also validation and testing of services and reusable service components. It proposes a methodology providing notations and usage guidelines to cover all stages during the development lifecycle of an arbitrary (telecommunication) service.

This paper presents means to test the functionality, scalability and performance of distributed telecommunication applications based on CORBA ORBs. It presents generic tools for testing the functionality, performance, robustness and scalability of distributed systems. The tools cover the test suite simulator TSsim for validating concurrent test suites, the TTCN/CORSA gateway TCgate for automated execution of the test cases (including a generic coder/decoder component), and the test manager TTman to setup and parameterize the test configuration and to control the test execution. The emphasis of this paper is on TTman and its scripting facilities. Exemplarily, a TINA access session which is part of the TINA platform developed by GMD FOKUS is considered.

This paper introduces our ongoing development of a network node architecture, which meets the requirements of dynamic run-time installation and configuration of network services. The objective is to provide a network service run-time system, which is independent from installed services and suitable for effective deployment of existing, new and future services. By developing network service specific resource abstractions, the architecture will be more flexible than existing approaches.

The TINA architecture specifications are well-known and have been available for many years. Experiences have been gathered with prototypical implementations of the TINA architecture or parts of it. These implementations have shown the benefits of the generic concept of TINA for surrounding distributed telecommunication services. Most of the implementations use new object-oriented implementation languages like C++ or Java and communication architectures like CORBA. However, up to now the question of the performance and satiability of TINA-conforming implementations, i.e., their eligibility for real applications with thousands of simultaneously acting users, still remains. This paper reports performance test results made with the TINA platform implementation at GMD FOKUS, which is based on C++, CORBA and Windows NT. The tests concentrate on the access session part of the service architecture (including subscription). The paper discusses the basic concepts of a flexible performance test architecture for distributed systems based on CORBA technology. Concrete performance test results are given, analysed and evaluated.

The paper discusses the need of test data filtering within traditional conformance testing methodology and advanced QoS data stream controlling. The common approach is the introduction of a data profile declaration. This profile declaration is useful for the (semi-)automatic generation of e.g. TTCN data constraints or object interface filters. The B-ISDN ATM Adaptation layer specification Type 1 is used as an application example in both application areas.

This paper shows that testing open object-based distributed systems is, in principle, feasible by applying the ISO/ITU-T conformance testing methodology framework. In this context, a gap in the (semi-) automatic test development and execution process has been identified and bridged. We have developed an approach to the application of the standardized test notation TTCN for testing ODL-based implementations. The equivalents of TTCN features informal TINA specifications have been identified. Further a TTCN-based test system implementation has been integrated in the CORBA environment. Test specification and execution are discussed w.r.t. the TINA retailer reference point.

Due to the highly increasing complexity of new telecommunication services and the distributed nature of them on the one hand and the requirement to come up with a short time to market on the other hand, new methods, techniques and tools covering the whole service creation process are needed. This paper presents an integrated approach covering the fields of designing, validating and testing services and reusable service components. For the design stage a methodology which applies concepts from the Reference Model for Open Distributed Processing (ODP) is introduced. To ensure that the service to be designed meets the requirements of a potential user, a validation stage has been included into the design methodology. This paper also contains a new method for automated testing of distributed services through executing these test cases in a CORBA based target environment to check whether or not the implementation fulfils the specification.

This paper discusses validation techniques for communication protocols and analyzes the practical use of selected validation techniques in an automated manner. A case study on validating the ATM (Asynchronous Transfer Mode) Signalling Protocol as specified by ITU-T in Q.2931 is used for this analysis. Different error classes are identified and valiyeard. An assessment of the different validation techniques in terms of effort and quality of results is given. As a result of the case study, an evaluation of the Q.2931 SDL specification completes the paper.

This paper discusses validation techniques for communication protocols and analyzes the practical use of selected validation techniques in an automated manner. A case study on validating the ATM (Asynchronous Transfer Mode) Signalling Protocol as specified by ITU-T in Q.2931 is used for this analysis. Different error classes are identified and validated. An assessment of the different validation techniques in terms of effort and quality of results is given. As a result of the case study, an evaluation of the Q.2931 SDL specification completes the paper.

This paper considers conformance testing of TINA service components in the realm of the ITU-T conformance testing methodology, which provides the only standardized, well accepted and widely used test methodology and test notation TTCN. It discusses the use of TTCN for testing computational objects of service components and proposes an implementation approach to derive executable tests from TTCN, which can be executed on any CORBA compliant ORB (with minor efforts for ORB specific adaptation) to check the functional correctness of deployed service components in distributed object environments. The TTCN/CORBA gateway is a general approach for testing distributed systems. TINA is taken as an example in this paper only. The work presented has been partially supported by the EC ACTS project TOSCA-TINA Open Service Creation Architecture, AC237.

Die Vergleichbarkeit von Zeit- und Kapazitätsmessungen bei Datenübertragungen erfordert eine eindeutige Beschreibung der durchzuführenden Experimente. In diesem Artikel stellen wir die Grundelemente für die Spezifikation solcher Leistungstests vor. Unser Ansatz basiert auf einer Erweiterung der bislang für Kommunikationsprotokolle einzigen standardisierten Testbeschreibungssprache TTCN. Wir zeigen, wie diese Erweiterung für die Definition und Implementierung verschiedener Leistungstests am AAL5 Protokoll eingesetzt wurde und diskutieren die beobachteten Meßergebnisse.

A generic test architecture for conformance, interoperability and performance testing of distributed systems is presented. The generic test architecture extends current test architectures with respect to the types of systems that can be tested. Whereas in the conformance testing methodology and framework the focus is on testing protocol implementations, the generic architecture focuses on testing real distributed systems whose communication functions are implemented on different real systems and whose correctness can only be assessed when tested as a whole. In support of the latter requirement, a test system itself is regarded as being a distributed system whose behaviour is determined by the behaviour of components and their interaction using a flexible and dynamic communication structure.

It is well known that the development of ATSs and ETSs is time and cost intensive. This contribution provides practical experiences and results gained from a test suite development project in the context of today’s ATM protocols. In particular we discuss the problems and effort to develop and execute the BISUP and MTP test suites within B-ISDN based on corresponding N-ISDN tests. The consideration of our approach and observations should help during the production and execution of related protocol conformance tests. We have outlined basic ideas on how to underpin methodologically the ATS migration process.

This paper presents a performance test suite for the ATM adaptation layer 5, that can be used to test the end-to-end performance of AAL5 implementations. The test suite includes throughput, frame loss ratio and goodput tests, frame latency tests, and maximum frame burst size tests. It is based on the performance testing white paper of the ATM Forum and shows the usability of PerfTTCN - a performance-extension of TTCN - for the description of performance tests.

This paper presents a new approach to test the performance of protocols, services, or application implementations (IUT) under normal and overload situations, to identify performance levels of the IUT for ranges of parameter settings and to assess the measured performance. A performance test describes precisely the configuration of the IUT, the configuration of the network, and the load characteristics (the so called performance test configuration). PerfTTCN - an extension of TTCN with notions of time, measurements, and performance - is a formalism to describe performance tests in an understandable, unambiguous and re-usable way with the benefit to make the test results comparable. 1. Motivation Different approaches for guaranteeing the end user certain levels of QoS were developed, since the non-functional aspects, in particular Quality-of-Service (QoS) aspects of distributed telecommunication services (e.g. multimedia collaboration, tele-teaching, etc.) are as important as functional aspects. They include approaches for QoS negotiation between peer user and service and network provider, QoS guarantees of communication services, QoS monitoring and management including self-adapting applications. This proposal considers QoS in the area of testing. In general, testing is a method to check whether an implementation meets certain requirements that are described in a specification. QoS testing checks the end-to-end quality of a service implementation against the QoS requirements. A specific class of QoS is that of performance-oriented QoS including delays (e.g. of a response), throughputs (e.g. for bulk data), and rates (e.g. of errors). We concentrate exclusively on performance-oriented QoS, other classes of QoS are out of the scope. Subsequently, we use the term perform...

This paper presents a new approach for extending Message Sequence Charts (MSCs) with time concepts. The approach is mainly aimed at using MSCs in the context of performance testing. The time extensions offer means to formally specify test purposes for performance testing as well as to describe time extended execution traces of distributed systems tests. Although already a number of time extensions has been proposed for MSCs, these are mainly aimed at describing timing requirements and not time measurements, which are needed for performance testing.

The paper discusses basic concepts to performance test a protocol implementation. Performance testing is understood as an extension to classical conformance testing. We present a methodology and some results of its working steps, that were obtained from applying it to the test of the ATM signalling protocol.

The paper describes an ongoing work on setting up a formal based testing methodology that covers the test suite generation from a protocol specification, the consistency check between protocol specification and test suite, the implementation of test suites, and the test result analysis. We are going to establish a tool set that allows us to automate the testing process as far as possible. The working example are ATM signalling protocols in local area networks.

The paper presents LotoTis – a performance-oriented and upward-compatible extension of Lotos, whose main feature are structured actions. A structured action is parameterized with an interaction time, a priority, a set of requested resources, and a monitoring signal. It allows us to model system tasks of distributed systems with their performance characteristics very easily. In addition, the paper compares LotoTis with related performance-oriented Lotos extensions and identifies the advantages and shortcomings of these approaches.

The paper presents the concept of structured actions for modeling performance aspects within process-algebraic calculi. A struc-tured action is parameterized with an interaction time, a priority, a set of requested resources, and a monitoring signal. The paper introduces the new performance-oriented LOTOS extension LOTOTIS and shows its use by specifying the Tick-Tock Case Study. Finally, it investigates the LOTOTIS upward compatibility with LOTOS.

The paper presents the performance-oriented, Lotos extension Loto Tis. Loto Tis allows us to specify performance-oriented behavior via quantified time, quantified nondeterminism, quantified parallelism, and action monitoring. It offers a set of refinement rules from Lotos to Loto Tis. Therefore, Loto Tis supports the standard conform development of performance-oriented specifications from existing Lotos specifications.

The paper presents a specification of the steam boiler system in Time Extended LOTOS as an example to describe real-time, hybrid systems containing parts with discrete and continuous behavior in a time extended process-algebraic formalism. The specification has been developed in three design steps – the specification of functional behavior in LOTOS, the specification of timed behavior in TE-LOTOS (a time extension of LOTOS), and the specification of data dependencies in TE-LOTOS with Gofer (a functional language). The resulting specification is modular, succinct, and easy to read. Most importantly, it is an implementation-near specification. It offers means for logical reasoning on the system properties, for stepwise refinement, simulation and prototyping of the system.

This paper examines some of the issues arising from customer requirements concerning the management of end-to-end services and guaranteed end-to-end quality of service. The implications of supporting the desired management capabilities both horizontally (inter-domain cooperative management) and vertically (from the service to the network elements) are discussed using as examples work currently being undertaken in two complementary projects.

Quality of service considerations have to be included in the whole protocol engineering cycle. The authors present an integrated approach for the design of communication protocols, including: performance enhanced formal specification of communication protocols, formal requirement specification, statistical performance analysis out of the enhanced formal specification, and verification of bounded performance properties out of the formal specification.