Workflow Research News

New version of Occopus cloud orchestrator

LPDS News - 11 April 2016 - 10:42am

We are happy to announce the 1.0 version of Occopus.

New features include:

  • Restructure node definition format
  • Introduce schema checking
  • Mixed config-manager support
  • Refactor plugin names
  • Reorganise config and authentication
  • New authenticator selection mechanism
  • New filtering mechanism in node description
  • Simplification of health_check
  • Introduce getip() in context templates
  • Update occopus commands
  • Introduce occopus-maintain command for maintenance
  • Introduce occopus-scale command for scaling
  • Support for multi infrastructure handling
  • Refactor occopus-import command parameters

Call for Participation - Advanced Course: High Performance Computations for Engineering

LPDS News - 31 March 2016 - 2:16pm
Advanced Course: High Performance Computations for Engineering
6th-10th June, 2016
Pecs, Hungary

Introduction:
Once again the successful course on HPC applied to Engineering will be held at the University of Pecs, Hungary. The course is suitable for novice and experienced users of HPC.

Course Objectives:
To provide an overview of the state of the art of high performance computing as applied to engineering applications. Special reference will be given to parallel, distributed and cloud computing and how serial or sequential algorithms for engineering problems may be parallelized for the efficient solution of large scale problems in engineering, analysis, simulation and design.

Course Lecturers:
Prof. Dr. Wolfgang Gentzsch, Co-founder and President, the UberCloud Prof. Dr. P. Iványi, University of Pécs, Hungary; Prof. Dr. J. Kruis, Czech Technical University, Czech Republic; Prof. Dr. B.H.V. Topping, Heriot-Watt University, UK and University of Pécs, Hungary.

For further details please see the webpage: http://hpc-course.mik.pte.hu/ or email: hpc-course@mik.pte.hu

Improved version of Occopus has been released

LPDS News - 18 January 2016 - 11:34am
The new, significantly improved version of Occopus has been released.

It has the following new features:
  • introduce periodical service health checking
  • new service health check mechanism: database check
  • new service health check mechanism: port check
  • add timeout for service unavailability
  • improved nova plugin: voms based authorization
  • new plugin: handling docker cluster
  • new plugin: occi cloud interface for EGI FedClouds
  • tutorials to demonstrate chef, docker and occi plugins
  • node definition 'synch_strategy' keyword renamed to 'service_health_check'


You can find more details about Occopus on the Occopus website.

First release of Occopus

LPDS News - 13 November 2015 - 1:59pm

SZTAKI is happy to announce the first official release of Occopus. Occopus is a framework that provides automatic features for configuring and orchestrating distributed applications (so called virtual infrastructures) on single or multi cloud systems. Occopus can be used by application developers and application controllers to manage virtual infrastructures at deployment time and at runtime as well as to create and provide complex virtual infrastructures to be built by one click even by novice cloud users.

The official web page can be visited at: http://occopus.lpds.sztaki.hu/

There are ready-to-use tutorials for Amazon and also for clouds via CloudBroker Platform.

Occopus can easily be connected (or integrated) to various applications via its REST interface.

For user support please visit the Occopus forum at http://community.lpds.sztaki.hu/ where for example you can get detailed information on how to use Occopus with Amazon.

We hope you will find useful this new tool by which complex virtual infrastructures could be deployed in clouds and we are looking forward to your comments and feedbacks.

IDGF Operation Centre became fully operational as part of the EGI grid infrastructure

LPDS News - 11 December 2014 - 11:18am
We proudly announce that after a long, but successful integration procedure the IDGF Operation Centre (OC) became fully operational and part of the EGI grid infrastructure. All necessary teams and tools are established in IDGF Operations Centre and ready for production. The integration procedure was done by MTA SZTAKI in Hungary. From now on, international volunteer grid infrastructures may join to the EGI grid infrastructure through the IDGF OC as a resource site and may offer their computational capacity for the European scientists. The IDGF OC opens the door for desktop grid and crowd computing infrastructures to support international research in many scientific disciplines.

New cloud project to start next February

LPDS News - 5 November 2014 - 2:58pm

A new 2 years long project, called ENTICE (dEcentralized repositories for traNsparent and efficienT vIrtual maChine opErations), will start on 1 February 2015. In this project, we will research and create a novel Virtual Machine image repository and operational environment named ENTICE for federated Cloud infrastructures aiming to:

  • simplify the creation of lightweight and highly optimised VM images tuned for functional descriptions of applications
  • automatically decompose and distribute VM images based on multi-objective optimisation (performance, economic costs, storage size, and QoS needs) and a knowledge base and reasoning infrastructure to meet application runtime requirements and
  • elastic auto-scale applications on Cloud resources based on their fluctuating load with optimised VM interoperability across Cloud infrastructures and without provider lock-in, in order to finally fulfil the promises that virtualization technology has failed to deliver so far.

Robert Lovas attending the EECA Cluster awareness raising and networking activity

LPDS News - 30 October 2014 - 4:15pm
Robert Lovas (deputy head of the laboratory) was invited to participate at the EECA Cluster awareness raising and networking activities event in Baku, Azerbaijan. The 3 days event is organised by the EECA Cluster in collaboration with the DG CNECT of the European Commission, the International Relations Office of the National Academy of Sciences of Azerbaijan and the Regional Innovative Technologies Academy (R.I.T.A.) on 12-14 November 2014. Mr. Lovas will introduce our work and projects during the meeting.

BOINC Workshop14 held at Budapest

LPDS News - 29 September 2014 - 11:52am

The 10th BOINC Workshop is held at the Laboratory of Parallel and Distributed Systems, in Budapest, Hungary. The program contains two days of presentations followed by a two days long hackfest event.

The workshop is for researchers, scientists and developers with significant experience or interest in BOINC. Areas of interest include:

  • BOINC on Android
  • GPU, multi-thread, VM-based, and Android applications
  • Data-intensive applications
  • Multi-user projects
  • Remote job submission
  • Integration with hubs, clouds, grids, and desktop grids

All participants will have the opportunity to present and discuss their work.

The full list of presentations can be found at the event's website at
http://boinc.berkeley.edu/trac/wiki/WorkShop14

Linux / Unix rendszergazda mellékállás

LPDS News - 11 September 2014 - 1:45pm
Az MTA SZTAKI Párhuzamos és Elosztott Rendszerek Kutatólaboratóriuma Linux rendszergazda pozíció betöltésére mellékállású részmunkaidős munkatársat keres XIII. kerületi központjába.

Feladatok:
  • Részvétel az ország egyik legkorszerűbb szuperszámítógépes rendszerének üzemeltetésében
  • Részvétel a SZTAKI felhő infrastruktúra fejlesztésében és üzemeltetésében
  • Linux szerverek üzemeltetése
  • Részvétel a központi és elosztott IT szolgáltatások telepítésében és felügyeletében

Elvárások:
  • Linux szerverek telepítése és karbantartása terén szerzett gyakorlat
  • Számítógép hálózati ismeretek
  • Alapvető hardveres ismeretek
  • Rendszerszemlélet, önállóság, fejlődőképesség, rugalmasság
  • Folyamatban lévő felsőfokú tanulmányok
  • Középfokú angol nyelvtudás

Előny:
  • Rendszeradminisztrátori gyakorlat
  • Jó programozási készség
  • Elosztott, magas rendelkezésre állású rendszerek üzemeltetésében szerzett tapasztalat
  • Képes legyen részlegesen specifikált feladat közel önálló elvégzésére
  • DevOps módszertan és eszközök ismerete

Felkínált lehetőségek:
  • Fiatal, dinamikus csapat
  • Szakmai előrelépési lehetőség
  • Részvétel hazai és nemzetközi projektek munkálataiban

Jelentkezési határidő: 2014. szeptember 29.
Pályázathoz kérjük csatolni a részletes szakmai önéletrajzot.
További információk és jelentkezés az alábbi elérhetőségeken: mellekallas-sysadmin@lpds.sztaki.hu, WEB: www.lpds.sztaki.hu

Linux / Unix rendszergazda álláshirdetések

LPDS News - 11 September 2014 - 1:18pm
Linux / Unix rendszergazda főállás Az MTA SZTAKI Párhuzamos és Elosztott Rendszerek Kutatólaboratóriuma Linux rendszergazda pozíció betöltésére főállású munkatársat keres XIII. kerületi központjába.

Feladatok:
  • Részvétel az ország egyik legkorszerűbb szuperszámítógépes rendszerének üzemeltetésében
  • Részvétel a SZTAKI felhő infrastruktúra fejlesztésében és üzemeltetésében
  • Linux szerverek üzemeltetése
  • Központi és elosztott IT szolgáltatások telepítése, felügyelete

Elvárások:
  • Linux szerverek telepítése és karbantartása terén szerzett magabiztos gyakorlat
  • Számítógép hálózati ismeretek
  • Alap hardveres ismeretek
  • Képes legyen részlegesen specifikált feladat közel önálló elvégzésére
  • Rendszerszemlélet, önállóság, fejlődőképesség, rugalmasság
  • Felsőfokú végzettség
  • Tárgyalóképes angol nyelvtudás

Előny:
  • Rendszeradminisztrátori gyakorlat
  • Jó programozási készség
  • Elosztott, magas rendelkezésre állású rendszerek üzemeltetésében szerzett tapasztalat
  • DevOps módszertan és eszközök ismerete

Felkínált lehetőségek:
  • Fiatal, dinamikus csapat
  • Szakmai előrelépési lehetőség
  • Részvétel hazai és nemzetközi projektek munkálataiban

Jelentkezési határidő: 2014. szeptember 29.

Pályázathoz kérjük csatolni a részletes szakmai önéletrajzot.
További információk és jelentkezés az alábbi elérhetőségeken: foallas-sysadmin@lpds.sztaki.hu,WEB: www.lpds.sztaki.hu

Memorandum of Understanding between ER-flow and XSEDE MoSGrid

ER-flow - 9 July 2014 - 3:51pm

ER-flow and XSEDE MoSGrid has signed a Memorandum of Understanding agreeing to work together for 3 years to achieve their common goals.

They agreed on to port the MoSGrid science gateway to the US XSEDE research infrastructure and to promote and disseminate the scientific gateway and to contribute in the dissemination of the ER-flow results. They will also promote and disseminate the workflow technology and workflow interoperability and give trainings on the gateway framework, workflow technology and workflow interoperability. Kitti Varga 2014-07-09T13:51:27Z

Memorandum of Understanding between ER-flow and STARnet

ER-flow - 12 May 2014 - 1:15pm

Last week the ER-flow project has signed an MoU with STARnet starting a collaboration on workflow sharing.

Kitti Varga 2014-05-12T11:15:58Z

New simulation platform release

ER-flow - 19 March 2014 - 2:11pm

We are releasing a new simulation platform which extends the production SHIWA Simulation Platform in a significant manner with new and upgraded services. To distinguish the current platform and the new one we call the old one as “production” and the new one as "prototype". To facilitate the transitional process both the current production platform and the new prototype platform will remain running in parallel for one month. The prototype platform's services are available at:

We deployed the prototype simulation platform to offer a new service, known as the prototype SHIWA Submission Service, and two updated services: the prototype SHIWA Portal and the prototype SHIWA Repository. These services will enable workflow developers and domain researchers to have greater control over the deployment of the applications and workflows they wish to use and their execution.

The new submission service is based on web services. It replaces the GT4 based GEMLCA Service. Having this submission service users can execute non-native workflows in a more reliable and secure way than with the GEMLCA Service. The prototype submission service allows execution of Kepler, MOTEUR and Taverna1/Taverna2 workflows. Further workflow systems support will be added later.

We support usage of the submission service in three phases. In phase 1 the submission service will be available only through the prototype SHIWA Portal. In phase 2 users can access it remotely from their community portal. In phase 3 it will be also possible to download and deploy the submission service locally and connect it to a community portal.

The prototype SHIWA Repository enables us to store all the information regarding workflow engines, their binaries/executables, attributes, parameters, and configurations. This includes information about different middleware and execution sites.

The database from the production SHIWA Repository has been ported to the prototype SHIWA Repository as of Friday 7th March at 9am. If you have completed any work on this repository since then, please contact an administrator to help you moving your work over to the new repository.

We also cleaned up the contents of the prototype SHIWA Repository. We kept all but a few workflows uploaded to the public domain. We archived and removed most of the tests, demo and training workflows. Please check the prototype SHIWA Repository. If you cannot find a workflow you need please contact Noam Weingarten.

The prototype SHIWA Portal was also upgraded to enable specifying the new submission service instead of the GEMLCA Service when creating workflow instances. In order to access the prototype SHIWA Portal we re-created the user accounts. Users registered with the production portal and repository registered will be receiving an email shortly with credential details for accessing the prototype new services.

We also upgraded the simulation platform's manuals adding the all the changes and upgrades. The upgraded manuals are available at:

To facilitate the transitional process both portals and repositories will remain running in parallel for one month. However we have to stress that any changes made on the production SHIWA Repository will not be maintained. So we urge you to use the prototype SHIWA Repository immediately.

If you have any question relating to the portal please contact:

  • SHIWA Submission Service
    • Benoit Meilhac at b.meilhac@westminster.ac.uk
  • SHIWA Repository
    • Edward Michniak at e.michniak@westminster.ac.uk
  • SHIWA Portal
    • Noam Weingarten at n.weingarten01@westminster.ac.uk
Kitti Varga 2014-03-19T13:11:03Z

SOMNO.NETZ and ER-flow has signed an MoU

ER-flow - 7 March 2014 - 1:07pm
The SOMNO.NETZ and ER-flow projects have signed a Memorandum of Understanding for collaboration in porting medical imaging applications as workflows to be executed on an OpenStack cloud infrastructure.   The SOMNO.NETZ project coordinator, Prof. Dr. Dagmar Krefting from the Technical University and the Charite Hospital in Berlin, has a long track of collaboration with the group of Dr. Silvia Olabarriaga of the Academic Medical Center of the University of Amsterdam, which participates in ER-flow. Dr. Krefting participated in the SHIWA project, and is familiar with the platform.    During the collaboration, the two groups will jointly investigate how to port medical imaging applications to a cloud infrastructure using WS-PGRADE. The first steps have been taken using the WS-PGRADE portal operated by SZTAKI, and now a new portal is being configured to access the German private cloud resources.   In the month of March 2014, a member of the Berlin team (Christof Jansen) is visiting the AMC for training on workflow development and portal configuration. He presented a talk at the AMC about the various German projects that will use this infrastructure for sharing and computing medical image data (see photo).     Kitti Varga 2014-03-07T12:07:20Z

SHIWA Technology on ECCB 2014

ER-flow - 3 March 2014 - 2:01pm

Our tutorial proposal has been accepted to the 13th European Conference on Computational Biology (ECCB'14). This tutorial will provide an introduction to the use of scientific workflows for bioinformatics data analysis, integration and scaling and will highlight the main differences and similarities between Galaxy, Taverna and WS-PGRADE systems.

ECCB'14 will be in Strasbourg, from September 7 to 10, 2014.

Kitti Varga 2014-03-03T13:01:21Z

New submission service to be introduced

ER-flow - 28 January 2014 - 12:13pm
The technology provider team of the ER-flow project has completed a major upgrade of the SHIWA Simulation Platform. The key component of the upgraded platform is the new web service based submission service which replaces the previous GT4 based GEMLCA Service.   As a result, to deploy and/or access remotely the submission service will be much easier than previously. There is a further advantage: Certificates are no longer required to access and use the submission service. The SHIWA Repository was expanded with workflow engine management which was done previously by the GEMLCA Repository. As a result, there will be only a single repository instead of the previous two ones. The SHIWA Portal was also upgraded to support the new submission service.    The development tests were completed successfully and the helio community will run the end user tests from 28th January using helio Taverna workflows. As soon as the tests are completed we will release the upgraded SHIWA Portal.  Kitti Varga 2014-01-28T11:13:04Z

New SHIWA Portal release

ER-flow - 21 January 2014 - 10:24am

The SHIWA Portal upgrade is completed and tested. The new portal is based on gUSE 3.6.1.

The principle changes are that the Workflow Editor will run on current versions of Java. The main improvement in gUSE v3.6.1 is the solution for debugging of job submissions by applying breakpoints in WS-PGRADE workflows. By this tool the users can interact with workflows at runtime: they can directly influence and change workflow’s submission processes and results by enabling or prohibiting the progress of job instances at breakpoints.

To access the SHIWA Portal, please use: http://ssp.shiwa-workflow.eu

Kitti Varga 2014-01-21T09:24:53Z

Success of LPDS PhD students

LPDS News - 28 November 2013 - 3:42pm

The annual report by the PhD students at MTA SZTAKI were organized a few weeks ago where the Laboratory of Parallel and Distributed Systems was presented by several members.

Based on the annual presentations, the importance of their research topic and the committed work of the past year, MTA SZTAKI has awarded the best students.

Now we proudly announce that LPDS member Ádám Visegrádi received the recognition of the Institute as the best valued first year student of the MTA SZTAKI.

The goal of Ádám Visegrádi’s research is to understand and solve the problems of distributed volunteer storage and data processing. Solving these problems would enable scientific communities to use desktop grids for new, data intensive applications, which was impossible before. Furthermore, the experience and results can be transferred to other distributed systems, including clouds.

The first results of this endeavour is the idea and the practical evaluation of using random linear coding in such volunteer storage scenarios. This paradigm enables more efficient utilization of network and storage resources. The future of this research is to design a working storage system for the BOINC desktop grid platform, that enables scientists and other users to run data intensive applications in the desktop grid.

Besides Ádám's success, two LPDS member, Ákos Hajnal (Practical Application of Program Slicing and Its Use in Software Testing and Maintenance) and Zoltán Farkas (A generic grid interoperability solution and discussion of related hierarchical scheduling questions) defended their doctorate this autumn.

Congratulations, Ádám, Ákos and Zoltán!

Zeta Search - a new application for SZDG

LPDS News - 21 November 2013 - 9:29am

Researchers in Eötvös Lóránd University (ELTE) within the The Riemann-zeta Search Project has been developing a new application for SZTAKI Desktop Grid (SZDG) for a while. Now we proudly announce that the Zeta search has been released and is available for everyone interested in.

Prime numbers are the building blocks of mathematics and affects every part of life. Primes can be found when you use your ATM card to buy something on Ebay, or send e-mails. The Riemann-Zeta function has an interesting property: the distribution of its zeros has the same behaviour as the distribution of primes.

The goal of the The Riemann-zeta Search Project project is to find very large values of Z(t), because we believe, that analysing these values can lead to a deeper understanding of primes, and help to unfold the mystery behind their distribution.

All the values and results found will be published on the website of the project: www.riemann-siegel.com.

Video about how astrophysicists are using the SHIWA Technology

ER-flow - 29 October 2013 - 3:18pm

A video is available showing how the Astrophysics community is using the SHIWA Technology.

Kitti Varga 2013-10-29T14:18:04Z

Pages

Subscribe to MTA SZTAKI LPDS aggregator - Workflow Research News