During the last years, the Web has emerged as a vast collection of resources that provide data both in the form of (static) documents and in the form of services. The huge number of available data resources in the Web, as well as ongoing trends of Web-based data processing such as service mashups, Data-as-a-Service or Web content integration portals call for techniques to aggregate (i.e., collect, select and transform) heterogeneous data from distributed sources in a uniform way.
We argue that there is still a lack of generic frameworks that support loosely coupled, distributed aggregation of heterogeneous data published in the Web. We present the WS-Aggregation framework as a generic solution to this problem. WS-Aggregation employs a set of interconnected aggregator nodes, which cooperate with each other to execute client requests. Following the SOA paradigm, WS-Aggregation decouples the aggregation platform from the target data services. The endpoint information of the system participants is stored in the VRESCo service registry and binding takes place dynamically at runtime.
The WAQL Query Language
WS-Aggregation makes use of the Web services Aggregation Query Language (WAQL), which is based on XQuery and provides a set of additional convenience features which we deem important in the context of data aggregation on the Web. The WAQL parser and preprocessor have been implemented with the aid of the parser generator JavaCC. A detailed description of the implementation can be found here.
Web User Interface
We have deployed a publicly available version of the WS-Aggregation Web user interface
here. You can use it to get acquainted with the features of WS-Aggregation.
The registration is free and fast - try it out now!
(System requirements: Mozilla Firefox, Flash Plugin)
Please consider a fair use policy. Please let us know if you find a bug or have any suggestions for improvement.
We offer several student projects and (unpaid) internships where you can contribute to this project (see our Open Topics page for details). If you are interested in contributing to this project or have any question concerning the project, do not hesitate to contact us.
Source Code and Examples
You can download the code (source and binary) here:
- nightly build (experimental!)
- WS-Aggregation-2012-04-30.zip (6.6MB) (minor fixes and improvements)
- WS-Aggregation-2012-04-28.zip (6.6MB) (latest version with user interface)
- WS-Aggregation-2011-10-15.zip (6.8MB) (migrated to Maven build)
- WS-Aggregation-2010-12-01.zip (15.6MB)
- WS-Aggregation-2010-06-20.zip (11.8MB)
Please read the README.txt for installation instructions.