Familiar Bedfellows | The S-CASE automation engine, Model Driven Engineering and Victorian buildings

In this blog post, the first of a series, Christopher Zolotas attempts to demonstrate the cornerstone technology used in S-CASE for automation, namely: Model Driven Engineering (MDE) using the analogy of Victorian architecture.

Object Management Group (OMG) has introduced MDE since 2000 and pretty much aspires to change the software design and construction paradigm. Its principal concept is the transition of code-centric software engineering to a model-based one, so as to achieve higher consistency, increased level of automation and productivity. In a nutshell, it aspires to deliver software in less time and at lower cost.

That said, the abstract nature of MDE concepts often makes it difficult to fully grasp the whole procedure. Therefore, in this blog post we will use an extensive metaphor taken from civil engineering domain through which we will gradually build up the concepts of MDE. Once this is done, in part two of this blog post we will examine the way S-CASE embeds MDE in order to automate the building process of RESTful services.

For the sake of this metaphor, we will use Victorian era buildings, which have a lot of distinct design characteristics and properties. Although there are quite many variations of Victorian Style buildings, we will examine the ones found in figure 1.

http://en.wikipedia.org/wiki/Victorian_house

Figure 1: Common Victorian style houses

Just by taking a look at the exterior of a Victorian house, we observe some common properties and patterns that differentiate it from all the others. For example, they have steep dark colored roofs and a lot of ornamentation details. Additionally, they are built with bricks and have a lot of fancy sash-windows with large glass panes.

Moreover, Victorian houses share a lot of common characteristics in their interior. Figures 2 and 3 illustrate two such examples. The wallpapers and the carpets are rich in patterns with either of them dyed in rich dark ruby reds or forest greens. The interior also exhibits heavily upholstered furniture that is usually overstuffed, while the woodwork is usually dark colored. Finally, when such houses were introduced, during the reign of Queen Victoria, they had gas-fuelled ceiling luminaries, gradually replacing the ones with candles.

http://en.wikipedia.org/wiki/Victorian_house

Figure 2: Demonstration of Victorian interior decorated with heavy patterned wallpapers

Of course, the point of this blog post is not to fully examine the Victorian style or even be absolutely accurate about it. The purpose is to clearly demonstrate that this specific style, as any other one, embeds some distinct characteristics and properties that differentiate it from the other ones. As a consequence, once you decide to buy such a house you expect it to have the aforementioned properties.  For the shake of this metaphor, let’s assume that we are hidden spectators of a conversation between some 19th century customer, who wants to buy a house, and a civil engineer. We will also assume that this civil engineer designs only Victorian houses. Furthermore, he will also play the role of a Victorian style decorator of the building interior. Therefore, since this customer picked this specific engineer, it means that he knows that what he is going to buy is a Victorian house with the aforementioned properties.

http://en.wikipedia.org/wiki/Victorian_house

Figure 3: Demonstration of a Victorian drawing room

During this conversation, the customer would probably explain his needs to the engineer. For instance he would probably want a spacious drawing room, a kitchen, a lavatory, a luminous dining room, one master bedroom and two child bedrooms, one for a boy and one for a girl. During this early stage of this collaboration however, no one considers any design issues or any specific materials that should be used for the building. They are focused on concepts related to the domain of house buildings. Thus, they are discussing only about the type and number of rooms it should have and probably some more details and properties of them. That’s it!

This is actually what happens during the first stage of MDE, which is the formulation of the Computational Independent Model (CIM). Nothing that has to do with the design of the actual piece of software is discussed, let alone specific implementation technologies or coding. It is only a discussion on what should be included in that house which is going to be built and nothing more.

Once the civil engineer is given the desired CIM concepts from his customer, he patiently and meticulously starts to design the building having in mind that the Victorian style must be applied to it. This is a step-by-step process that this civil engineer has to follow in order to create the design of the house in such a way that all the concepts his customer asked for in the CIM (3 bedrooms, 1 drawing room etc.) are taken care of. In particular, he will design a boy bedroom by applying to it Victorian properties like heavily patterned wallpapers and carpets, dark-coloured wood furniture for any closets/chest of drawers etc. and of course fancy sash-windows with big glass panes and a gas-fuelled ceiling luminaire!

However, the engineer does not yet bother to pick the specific materials with which the house will be built. He merely writes on his design that for the house exterior there will be used some sort of bricks. Additionally, the interior walls will be covered with some sort of patterned wallpapers, whilst the furniture will be of some type of dark-colored wood and the lighting will be of some sort of ceiling gas-luminaire. Full stop! Once he is done applying the Victorian style to every concept his customer asked for in CIM, he knows that he is done with the second phase of MDE, which is the Platform Independent Model (PIM) formulation or more appropriately the transformation of the CIM to its according PIM. In other words, in this second phase the design of the whole house that satisfies the customer needs, which were prescribed in the CIM, is introduced. Full stop!

Once the engineer is done with the PIM and before he calls his construction team to build the house, he still has to decide the specific materials to be used for that specific house. How does he decide that? This information originates principally from his customer. The customer will have to pick a specific combination of material brands that his engineer is able to handle. He is the one who pays after all!

http://en.wikipedia.org/wiki/Victorian_house

Figure 4: Some parts, like a garden, will need specialised personnel like a gardener.

Once the engineer knows his customer selection, he is able to proceed to the next phase. At this point he knows that the parts of the building that consist of bricks should be build with bricks of make X, the wallpapers should be of make Y, the ceiling gas lighting should be a specific 1897 model of make Z etc. In fact, when this point is reached, the engineer knows that he has transformed every single component of the house PIM to its specified counterpart, which is the Platform Specific Model (PSM). In other words, the engineer has specialized the materials with which the building of that specific design described in PIM is going to be implemented according to one of the available material combinations. Other customers could possibly pick other material combinations though (other PSMs). Thus, there is a relationship of one PIM to many PSMs. One PIM that embeds a specific design may be transformed to any number of different PSMs (different combinations of specific materials).

The final step is to employ a specific construction team that knows how to construct the Victorian buildings with the specific kind of materials that were selected. For example, he will have to hire builders that know how to handle bricks of make X following the Victorian Style. Similarly, he will hire an interior decorator that knows how to apply on the walls make Y wallpapers that are heavily patterned etc. Of course, it is quite probable that some parts of the building may be partially completed or even empty. For example, parts like a garden (figure 4) will need specialized personnel, like a gardener, to get completed. This final step is the analogous phase in MDE of the code generation from a PSM. Some parts of the code will be fully implemented, whilst others will need programmer intervention to get completed.

This is pretty much MDE. It comprises of four phases through which the MDE engine (civil engineer) has to go through. The CIM formulation with the high level domain concepts, then the transformation to the PIM in order to introduce the appropriate design, followed by the transformation to a PSM where specific materials are picked up for every aspect of the subject that is going to be built. The final step is the code generation (the action of building) and the revision by developers (specialized experts). It must be stressed though, that this procedure is automated. That’s the point! For instance, if a customer was able to speak to an imaginary MDE engine that knows how to build Victorian houses, he would say his needs to it and then the MDE engine would return to him the building automatically and instantly! In other words, the transformations, transitions from phase to phase, are automated.

In the following blog post, the second part of the MDE introduction, we will build upon the concepts demonstrated in this one through this metaphor. Thus, we will attempt to show how S-CASE embeds MDE in order to build RESTful web services or in other words, pieces of software that follow the REST architectural style.

S-CASE Blog | The S-CASE concept

In our latest blog entry, project technical coordinator, Kyriakos Chatzidimitriou, takes us through the world of S-CASE, highlighting the project components and demonstrating how S-CASE will be realised.

The S-CASE project is about semi-automatically creating RESTful Web Services through multi-modal requirements using a Model Driven Engineering methodology. The world of web services is moving towards REST and S-CASE aims at facilitating developers implementing such web services by focusing mainly on requirements engineering. The figure below depicts the basic components and basic flow of events/data in S-CASE.

 

In order to better understand the practical application of the S-CASE solution, let’s take a look at a typical use case example.
Through the S-CASE IDE the user imports or creates multi-modal requirements for his/her envisioned application. The requirements may be:
  • Textual requirements in the form “The user/system must be able to …”,
  • UML activity and use case diagrams created in the platform or imported as images,
  • Storyboards for flow charting, and
  • Analysis class diagrams to improve the accuracy of the system to identify entities, their properties and their relationships.

The requirements are then processed through natural language processing and image analysis techniques in order to extract relevant software engineering concepts. These are mainly the identification of RESTful resources, their properties and relations and Create-Read-Update-Delete (CRUD) actions on resources. All these concepts are stored in the S-CASE ontology.
The above procedure also identifies action-resource tuples that can be created automatically by the system like the action-resource “create bookmark” (automatically built) or others that need more elaborate processes like “get the weather given geolocation coordinates” (semi-automatically build or composed). The latter are send into the Web Services Synthesis and Composition module.

The Web Services Synthesis and Composition module tries to synthesize elaborate processes by composing 3rd party web services into a single S-CASE composite web service. To perform such a computation, S-CASE provides a methodology for semantically annotating 3rd party web services using S-CASE domain ontologies, so that they can later be matched to the requirements of the composite service. The composite service is deployed to the YouREST deployment environment and registered in the directory of S-CASE web services for future reference and re-use.

Upon completing the stages above, the model driven engineering procedure initiates. The first step is to create the Computational Independent Model (CIM) out of the S-CASE ontology. The CIM contains the bare minimum information needed to scaffold a REST service that adheres to the requirements imposed by the user, i.e. it includes all the problem’s domain concepts.

After that model transformations take place transforming the CIM into PIM (incorporate design constraints, but platform independent) and PSM (Add support for implementing the PIM into a specific suite of software tools like: java, jax-rs, hibernate, json, jaxb, postgresql etc.). The final step is to automatically generate the code of the web service. Calls to composite services are wrapped inside the generated code. The code is build and deployed to YouREST for others to use.

In order to support software re-use, every software artefact created from this procedure is stored into the S-CASE repository for future retrieval.

Through S-CASE we plan to develop an ecosystem of services, along with the appropriate tools for service providers to develop quality software for SMEs with an affordable budget.

S-CASE Blog | Natural Language Processing

We recently got another research paper on our work in S-CASE accepted  at a conference on natural language processing. The accepted paper describes our efforts on improving a parsing model that can  automatically map software requirements written in natural language to formal representations based on semantic roles.

State-of-the-art semantic role labelling systems require large  annotated corpora to achieve full performance. Unfortunately, such  corpora are expensive to produce and often do not generalise well  across domains. Even in domain, errors are often made where syntactic  information does not provide sufficient cues. In this paper, we mitigate both of these problems by employing distributional word representations gathered from unlabelled data. The rationale for this  approach lies in the so-called distributional hypothesis by Zellig Harris, which states that words that occur in the same contexts tend  to have similar meanings.

While straight-forward word representations  of predicates and arguments have already been shown to be useful for  semantic analysis tasks, we show that further gains can be achieved by composing representations that model the interaction between predicate  and argument, and capture full argument spans.

S-CASE Blog | S-CASE & ETICS: Infrastructure Pilot Application

By using S-CASE tools, Engineering will develop ETICS Build and Test Environment Manager (BTE Manager).

ETICS BTE is one of the three pilot applications, foreseen by the project, which will help partners validate and evaluate the S-CASE paradigm. ETICS (etics.res.eng.it) is a system which automates and improves the execution of builds and tests and verifies the quality of the software produced. It is especially conceived for distributed, multi-language and multi-platform software.

A build/test lifecycle cannot exclude the management of the environment where builds and tests are performed. In ETICS, at the moment, this is done manually by the developer or tester. The BTE manager, developed by using S-CASE, will be able to create/modify/delete, accordingly to users’ needs, the Virtual Machines of a certain Virtual Infrastructure. It will also optimize the resource consumption by reusing inactive virtual machines for specific build sessions and by quickly disposing useless machines. User’s will no longer need to manually create the needed virtual machines, profile them (adding, for example, Application Servers, Containers etc), tune the environment (e.g. managing the IP addresses, DHCP Servers etc) and, after the development or tests, dispose the whole environment.

Introduction & Overview to ETICS

ETICS (E-infrastructure for Testing, Integration and Configuration of Software) is a system which automates and improves the execution of builds and tests. It is especially conceived for distributed, multi-language and multi-platform software and provides meaningful measurements of the overall software quality. ETICS consists of a build and test execution system, offered as web-services, able to exploit distributed computational resources and a plug-in mechanism for integrating engineering tools, to design, maintain and monitor build and multi-node testing scenarios.

A complete build/test lifecycle cannot exclude the management of the environment where builds and tests are performed. Currently ETICS don’t bother of this aspect, which is manually performed by the developer or tester. In particular he/she has to create the needed virtual machines, profile them (adding, for example, Application Servers, Containers etc), tune the environment (e.g. managing the IP addresses, DHCP Servers etc) and, after the development or tests, dispose the whole environment.

The automation and the optimization of that process would be a very good improvement for Engineering’s development process. For this purpose a good solution may be a web service that receives as input the features of the requested environment and creates/modifies/deletes according the Virtual Machines of a certain Virtual Infrastructure. It may optimize the resource consumption by reusing inactive
virtual machines for specific build sessions and quickly dispose useless machines.

BTE Manager defines the needed VMs on the basis of the request received from ETICS
and of the current status of the requested environment (e.g. is it already present? Do we need new Vms? Do we need to destroy some of the VMs?). In some cases it tries to optimize the resource consumption by reusing virtual machines belonging to different environments. Obviously such an optimisation is not always appropriate: for example a test environment, where performance evaluation is probably requested, cannot be built with machines shared with other experiments.

We can distinguish two main flows: one for creation and management of virtual environments and one for disposal. Each flow in turn is divided into flows related to the specific environment: build or test.

For example let’s consider the flow concerning the creation of a virtual build
environment:

1. ETICS send the request that is received by the RESTful interfaces
2. The request is a creation
3. The requested environment is build
4. The module checks if the same virtual environment exists: in that case the requested modifications will be evaluated, otherwise the request will be considered a simple creation
5. The module evaluates if there is any active VM belonging to other build environments, that can be used also for this build activity. For example a machine containing a web server may also host a DBMS
6. The exact number and features of needed VMs is evaluated, the appropriate commands are generated and sent to the target Infrastructure
7. The response is parsed and a final response is sent to Etics.

The steps requested by the other use cases are very similar: the only remark is that the creation or disposal of a test environment does not include the re-use (or the preservation) of VMs requested in other operations.

Build and Test Environment Manager

ETICS Build and Test Environment Manager will be built by S-Case. It will expose some web services to interact with ETICS and will use some web services to interact with the Virtual Infrastructures.

The exposed web service should take as input the features of the requested
infrastructure. At high level it should provide the following methods:

• Create test environment
• Create build environment
• Modify test environment
• Modify build environment
• Dispose test environment
• Dispose build environment

Every build or test environment should be identified by an ID related with the name of the ETICS project: every management operation on a certain environment can be transparently performed by ETICS using the ID and a set of parameters

The central part of the activity, concerning the BT Environment Manager, describes the logical steps needed to generate the final request for the infrastructure: these functionalities are provided by the Core. In particular it decides if:

• The build or test environment requested exists
• Some VM have to be added or removed
• Some VM can be reused to optimize the resource consumption

These operations may involve matchmaking algorithms and produce a request for a specific web service managing a specific Virtual Environment (for example, Windows Azure).

The web client sends the request, that can be a SOAP or REST request with the operation that the Virtual Environment Manager must perform to create, delete or adapt the Environment for the operation requested.

Requirements

Functionalities

• Must translate the requests from ETICS in commands for the Virtual Infrastructure
◦ This could be accomplished by using a local database
◦ As an alternative by talking with the Infrastructure
• Must know the software installed on each Virtual Machine
• Must be able to create/delete/modify with granularity from environment to single machines Interfaces
• Must expose RESTFul API by which ETICS will be able to request the environments
• It should be possible to obtain information on active environments through RESTFul API
• Must interact with Infrastructure Management API of a set of infrastructures

Third party Services

The Module to be produced will be able to interact with different Virtual Infrastructures. Actually CLOE, Engineering Virtual Infrastructure, will be the most important one, but, since in several use cases Engineering developers need to use build and test environments on other Virtual Infrastructures, it will be very useful to support the most widespread.

A first list is the following:

• Amazon
• OpenStack
• Microsoft Azure.

These Virtual Infrastructures (IaaS) exposes APIs enabling to manage their Virtual Machines. The management of the software on the VMs is more complex and, in general, depends on the VMs.

• Must keep track of the features of the generated environments

A complete software to manage Virtual Infrastructures and to profile Virtual Machines is Foreman (http://projects.theforeman.org/): it exposes a complete set of RESTful APIs to interact with several Virtual Infrastructures (including those in the list) and to profile
the machines.

Foreman’s RESTful API should be added to the list above in order to have a complete and flexible approach to manage several Virtual Infrastructures.