The Grid Economy Project

Economic Paradigm for "Resource Management and Scheduling" for Service-Oriented Grid Computing

Grid Computing Info Centre

Updates:

The Result of this R&D: A PhD thesis on Economic Paradigm for Grid Computing
All future work on grid economy and new research threads leading from this work will be carried out through the Gridbus Project.

The focus of our research is on the development of economic or market-based resource management and scheduling system for global grid computing. The research and development is being carried in two threads:

GRid Architecture for Computational Economy (GRACE)
Grid Resource Broker (GRB, aka Super Schedulers/MetaSchedulers)

The rationale for using economics as a metaphor for management of resources and scheduling in peer-to-peer and grid computing along with scientific results are discussed in EuroGlobus invited/keynote talk:

Grid Meets Economics (also Power Point version)
Audio version of PPT slides: Nimrod-G and Virtual Lab Tools for Data Intensive Computing on Grid: Drug Design Application Case Study

GRid Architecture for Computational Economy (GRACE)

We are exploring the development of a generic framework/infrastructure for grid computational economy called GRACE that co-exist with Grid systems such as Globus and Legion for creating an Economy Grid. The key components of GRACE infrastructure include:

Grid Resource Broker (e.g., Nimrod/G)
Grid Resource and Market Information Server (provided by Globus MDS and extended services for resource access price)
Grid Open Trading Protocols and API
Trade Manager (part of broker involved in establing service price)
Grid Trade Server (works for Grid Service Providers)

They work closely with grid middleware and fabrics. The GRACE-infrastructure supports generic interfaces (protocols and APIs) that can be used by the grid tools and applications programmers to develop software supporting the computational economy.

Grid Resource Broker (GRB)

The Nimrod-G Resource Broker is a grid application scheduler and it is responsible for resource discovery, selection, scheduling, and deployment of computations over them. The computational model of applications to be scheduled on Grid can be sequential, embarrassingly or plesantly parallel (task farming), and parallel in nature. Applications can be parameterized (e.g., shop simulation, Drug Design, network and fuzzy logic simulation, high energy physics events processing) using our simple, Parameter Sweep Specification (PSP) language for SPMD (single program, multiple data) style data parallel computing.

We are addressing scheduling of parameter sweep (task farming) applications on World Wide Grid (WWG) resources as part of the Nimrod/G project. The previous version of Nimrod/G supported deadline based scheduling by keeping the cost of computation as low as possible. The latest version supports both deadline(soft realtime) and budget (computational economy) constraits in scheduling and at the same time it can optimise execution time or budget expenses. The GRACE infrastructure will enable Nimrod/G to dynamically trade for grid resources in the open market environment and select resources that meet user requirements (deadline and cost). Our focus is on the the design and development of smart scheduling algorithms that support constraint-based scheduling. Our economics and soft-realtime scheduling algorithms explicity handle: existence of heterogenous and unreliable resources in Grid with varying performance, cost, price, acceess policies, and user constraints such as deadline for completion of the assigned work, and budget limitations. Basically, on behalf of users, broker smartly leases distributed resources to run applications depending on the QoS demands and yet meet user constraints.

Our Nimrod/G resource broker Framing Engine is a persitent and programmable component. It provides protocols and APIs for developing Nimrod/G clients for computational steering (e.g., Nimrod-G Monitor), customised applications for parametric computing (e.g., Active Sheets), and implementing user-level job schedulers.

Download Nimrod/G Resource Broker (now you can also get source code!).

Grid Simulation (GridSim) Toolkit

GridSim Toolkit for Modelling and Simulation of Scheduling for Parallel and Distributed Systems

Application Initiatives

Virtual Laboratory: Molecular Modelling for Drug Design on Peer-to-Peer World-Wide Grid
High Energy Physics and Grid Networks(HEPGrid) (under construction).

The Team

The research is carried out by Rajkumar Buyya in collaboration with David Abramson, Jonathan Giddy, and many external collaborators.

The World Wide Grid (WWG) testbed

You are invited to volunteer (idle resources of) your computers to this testbed.

In News and Press

Grid, P2P and Distributed Computing, Network Magazine, Feb. 2001.
New (Computer) Power to Solve World's Problems, Monash News, Volume 4, Number 3, April 2001, Australia.
Economics Driven P2P/GRID-Based Service Oriented Computing, HPCwire: The Electronic News Magazine for High Performance Computing, Vol. 10, No. 30, July 27, 2001, USA.
Networks: IBM enmeshed in grid computing, Red Herring, September 21, 2001.
Tools automate computer sharing, Technology Research News (TRN), September 12, 2001, USA.
Technology Research News Mag. interview on Economics of P2P/Grid computing
Harnessing a global grid, The Age News: From the World of Information Technology, October 9, 2001, Australia.
Technology Research News Mag. interview on the Virtual Laboratory project
GridSim Releases Java-based Grid Computing Toolkit, The Grid Computing Planet News, January 4, 2002, USA.
Toolset teams computers to design drugs, Technology Research News (TRN), January 16, 2002.
Distributed Computing Newsgroup Approved, Internet.Com News, March 1, 2002, USA.
Peer (to Peer) Pressure: It's a Good Thing, Software Development Magazine, USA, April 2002.
KaZaA users brace for hijack, The Age and the Sydney Morning Herald Newspapers, April 30, 2002. (local copy)

Publications

Rajkumar Buyya, David Abramson, Jonathan Giddy, Nimrod/G: An Architecture for a Resource Management and Scheduling System in a Global Computational Grid, The 4th International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2000), May 2000, Beijing, China. IEEE Computer Society Press, USA.
Rajkumar Buyya, David Abramson, Jonathan Giddy, An Economy Driven Resource Management Architecture for Global Computational Power Grids , The 2000 International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 2000), Las Vegas, USA, June 26-29, 2000.
Mark Baker, Rajkumar Buyya, Domenico Laforenza, The Grid: International Efforts in Global Computing, International Conference on Advances in Infrastructure for Electronic Business, Science,and Education on the Internet (SSGRR 2000), l`Aquila, Rome, Italy, July 31 - August 6. 2000.
Rajkumar Buyya, Jonathan Giddy, David Abramson, An Evaluation of Economy-based Resource Trading and Scheduling on Computational Power Grids for Parameter Sweep Applications

HPDC 2000

Ajith Abraham, Rajkumar Buyya, Baikunth Nath, Nature's Heuristics for Scheduling Jobs on Computational Grids, The 8th IEEE International Conference on Advanced Computing and Communications (ADCOM 2000), December 14-16, 2000, Cochin, India.
Klaus Krauter, Rajkumar Buyya, and Muthucumaru Maheswaran, A Taxonomy and Survey of Grid Resource Management Systems
Rajkumar Buyya, Steve Chapin, David DiNucci, Architectural Models for Resource Management in the Grid, The First IEEE/ACM International Workshop on Grid Computing (GRID 2000), Springer Verlag LNCS Series, Germany, Dec. 17, 2000, Bangalore, India.
Rajkumar Buyya, Jonathan Giddy, David Abramson, A Case for Economy Grid Architecture for Service-Oriented Grid Computing, 10th IEEE International Heterogeneous Computing Workshop (HCW 2001), In conjunction with IPDPS 2001, San Francisco, California, USA, April 2001.
Rajkumar Buyya and Sudharshan Vazhkudai, Compute Power Market: Towards a Market-Oriented Grid, First IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2001), Brisbane, Australia, May 16-18, 2001.
Rajkumar Buyya, Nimrod/G Problem Solving Environment and Computational Economies, Grid Computing Environments Community Practice (CP) Document, First Global Grid Forum (GGF) and European Datagrid Conference, Amsterdam, The Netherlands, March 4-7, 2001.
R. Buyya, T.Edison et. al., Problem Solving Environments: A Comparison Whitepaper, First Global Grid Forum (GGF) and European Datagrid Conference, Amsterdam, The Netherlands, March 4-7, 2001.
Mark Baker, Rajkumar Buyya, and Domenico Laforenza, Grids and Grid Technologies for Wide-Area Distributed Computing , Software: Practice and Experience (SPE) Journal, Wiley Press, 2002.
Rajkumar Buyya, Heinz Stockinger, Jonathan Giddy, and David Abramson, Economic Models for Management of Resources in Peer-to-Peer and Grid Computing, Technical Track on Commercial Applications for High-Performance Computing, SPIE International Symposium on The Convergence of Information Technologies and Communications (ITCom 2001), August 20-24, 2001, Denver, Colorado, USA.
Klaus Krauter, Rajkumar Buyya, and Muthucumaru Maheswaran, A Taxonomy and Survey of Grid Resource Management Systems for Distributed Computing

SPE

Madhu Chetty and Rajkumar Buyya, Weaving Computational Grids: How Analogous Are They with Electrical Grids?, Computin g in Science and Engineering (CiSE), The IEEE Computer Society and the American Institute of Physics, USA, July-August 2 002.
Manzur Murshed and Rajkumar Buyya, Using the GridSim Toolkit for Enabling Grid Computing Education, International Conference on Communication Networks and Distributed Systems Modeling and Simulation (CNDS 2002), January 27-31, 2002, San Antonio, Texas, USA.
Rajkumar Buyya and Manzur Murshed, GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing, The Journal of Concurrency and Computation: Practice and Experience (CCPE), Wiley Press, USA, May 2002.
Rajkumar Buyya, David Abramson, and Jonathan Giddy, Nimrod-G Resource Broker for Service-Oriented Grid Computing, IEEE Distributed Systems Online, in Volume 2 Number 7, November 2001.
Rajkumar Buyya, David Abramson, and Jonathan Giddy, A Computational Economy for Grid Computing and its Implementation in the Nimrod-G Resource Broker, Future Generation Computer Systems (FGCS Journal, Elsevier Science, The Netherlands, 2002.
Rajkumar Buyya, Kim Branson, Jon Giddy, and David Abramson, The Virtual Laboratory: Enabling Molecular Modelling for Drug Design on the World Wide Grid, Technical Report, Monash-CSSE-2001-103, Monash University, December 2001.
Rajkumar Buyya, Kim Branson, Jon Giddy, and David Abramson, The Virtual Laboratory: A Toolset for Utilising the World-Wide Grid to Design Drugs, 2nd IEEE International Symposium on Cluster Computing and the Grid (CCGrid 2002), 21 - 24 May 2002, Berlin, Germany.
Rajkumar Buyya, David Abramson, Jonathan Giddy, and Heinz Stockinger, Economic Models for Resource Management and Scheduling in Grid Computing, Special Issue on Grid Computing Environments, The Journal of Concurrency and Computation: Practice and Experience (CCPE), Wiley Press, USA, May 2002.
Rajkumar Buyya, Manzur Murshed, and David Abramson, A Deadline and Budget Constrained Cost-Time Optimization Algorithm for Scheduling Task Farming Applications on Global Grids, Technical Report, CSSE-2002/109, Monash University, Melbourne, Australia.

Presentations

Grid Scheduling and Computional Economy: Nimrod/G Perspective, High Assurance Laboratory, Dept. of Computer Science, Arizona State University (ASU), Tempe, USA, Nov. 22, 1999.
Computational Grids and Computational Economy: Nimrod/G Approach, Poster Session, IEEE International Workshop on Cluster Computing (IWCC'99), Melbourne, Australia, Dec. 2-3, 1999.
Grid Resource Management, Scheduling, and Computational Economy, 2nd International Workshop on Global and Cluster Computing (WGCC 2000), Tsukuba/Tokyo, Japan, March 15 - 17, 2000.

short notes

Economic-based Resource Management and Scheduling for Global/Grid Computing, CSSE Clayton Seminar Series, Monash University, Melbourne, April 10, 2000.
Nimrod/G: An Architecture for a Resource Management and Scheduling System in a Global Computational Grid, International Conference on High Performance Computing in Asia-Pacific Region (HPC Asia 2000), Beijing, China, May 15-17, 2000.
Grid Computing Technologies and International Efforts, Huazhong University of Science and Technology (HUST), Wuhan, China, May 19, 2000.
An Economy Driven Resource Management Architecture for Global Computational Power Grids, The University of Hong Kong, Hong Kong, May 24, 2000.
Seamless, Scalable Computing from Desktops to Global Computational Power Grids: Hype or Reality ?, Keynote Talk, International Conference on Parallel and Distributed Computing, Applications and Technologies (PDCAT 2000), Hong Kong, May 22-24, 2000. (Access: presentation slides).
The Application of PDC (Parallel and Distributed Computing) and Technique in E-Commerce, PDCAT 2000 Conference Panel, Hong Kong, May 22-24, 2000. (Access: presentation slides)
Internet-wide Global Supercomputing, AUUG 2000 Conference: Enterprise Security, Enterprise Linux, Canberra, 25-30 June 2000.
An Evaluation of Economy-based Resource Trading and Scheduling on Computational Power Grids for Parameter Sweep Applications

HPDC 2000

An Economic-based Resource Management and Scheduling for Computational Grids, IRISA / INRIA Rennes, Campus Universitaire de Beaulieu, France, Sept. 4, 2000.
An Economic-based Resource Management and Scheduling for Computational Grids, University of Paris-Sud, Paris, France, Sept. 5, 2000.
The Grid: A .COM or Service Oriented Software Cabling Technology for Building the Internet-scale CPU, The Gippsland School of Computing and Information Technology, Monash University (GSCIT), Gippsland Campus, Churchill, Australia, Oct. 10, 2000.(Access: presentation slides)
"Economy Grid: A New e-Paradigm for Grid/Internet Computing", Distributed Systems Technology Centre (DSTC), Monash University, Melbourne, Australia, Oct. 27, 2000. (Access: presentation slides)
Architectural Models for Resource Management in the Grid, First IEEE/ACM International Workshop on Grid Computing (GRID 2000), Dec. 17, 2000, Bangalore, India. (Access: presentation slides)
Emerging Trends in Internet Computing, IEEE Computer Society Hyderabad Section and Birla Science Centre, Hyderabad, India, Dec. 21, 2000. (Access: presentation slides)
Internet Computing and the Emerging Grid, Deakin University, Melbourne, Australia, March 9, 2001.
An Economics Paradigm for Peer-to-Peer (P2P) Grid Computing, Dept. of Computer Science, Mississippi State University, Starkville, USA, April 20, 2001.
A Case for Economy Grid Architecture for Service-Oriented Grid Computing, 10th IEEE International Heterogeneous Computing Workshop (HCW 2001), In conjunction with IPDPS 2001, San Francisco, California, USA, April 2001. (Access: presentation slides)
Grid meets Economics: A Market Paradigm for Resource Management and Scheduling for Peer-to-Peer, World-Wide Grid Computing, Sun Microsystems Inc., Palo Alto, California, USA, April 27, 2001.
Nimrod/G and Grid Market: Price is what you pay and the value is what you get!, Poster Exhibit, First IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGrid 2001), Brisbane, Australia, May 15-18. 2001.
A Market Paradigm for Solving Large-scale Data-Intensive Applications on Peer-to-Peer Grids, CSSE @ Monash University, Melbourne, Australia, May 28, 2001.
Grid meets Economics: A Market Paradigm for "Resource Management and Scheduling" for World-Wide Grid Computing, First EuroGlobus Workshop, Robinson Club Apulia Village, Lecce, Italy, June 16-23, 2001.(html version.)
Virtual Laboratory: Data Intensive Science during Holiday @ Robinson Village in Italy!, First EuroGlobus Workshop, Robinson Club Apulia Village, Lecce, Italy, June 16-23, 2001. (html version.)
Economic Models for Management of Resources in Peer-to-Peer and Grid Computing, Technical Track on Commercial Applications for High-Performance Computing, SPIE International Symposium on The Convergence of Information Technologies and Communications (ITCom 2001), August 20-24, 2001, Denver, Colorado, USA. (PowerPoint slides)
Economics Paradigm for "Resource Management and Scheduling" for Service Oriented P2P/Grid Computing, Joint Seminar of the Department of Electrical and Computer Engineering; and the Dept. of Computer Science, Colorado State University, Fort Collins, Colorado, USA, August 23, 2001. (Slides in: PDF or PPT format)
Economics Paradigm for Service Oriented Peer-to-Peer and Grid Computing, The School of Computer Science and Information Technology, The RMIT University, Melbourne, Australia, September 13, 2001. (Slides in: Power Point format)
Nimrod-G and Virtual Lab Tools for Data Intensive Computing on Grid: Drug Design Application Case Study, BioGrid Computing Symposium 2001, 26 October 2001, Singapore. Via Teleconference with the Audio version of PPT slides and pure PPT slides.
Nimrod-G Resource Broker for Service-Oriented Grid Computing, SC Global, Australian Node, November 14, 2001, Sydney. (ApGrid Panel, SC Global)
The Virtual Laboratory: Enabling Molecular Modeling for Drug Design on the World Wide Grid, SuperBioGRID Computing Symposium 2001, National University of SingaporeSingapore, Dec. 10, 2001. Via Teleconference with the Audio version of PPT slides.
Weaving the World Wide Grid Marketplace: Economic-based Distributed Resource Management and Scheduling for Service-Oriented Grid Computing, Commercial High Performance Computing: Conference and Expro, Kissimmee, Florida, USA, March 19-20, 2002. (presentation slides).
Weaving the World Wide Grid Marketplace: Economic-based Distributed Resource Management and Scheduling for Service-Oriented Grid Computing, IBM T. J. Watson Research Centre, New York, USA, March 21, 2002. (presentation slides).

Demonstrations

Nimrod/G - GRID Resource Broker and Computational Economy, Ninth International Symposium on High Performance Distributed Computing (HPDC 2000), Pittsburgh, USA, August 1-4, 2000.
Snaps of Economy Grid posters presentation @ CCGrid 2001: First IEEE/ACM International Symposium on Cluster Computing and the Grid, Brisbane, Australia, May 15-18. 2001. Photo Courtesy: Satoshi Matsuoka!
Nimrod-G: Economics driven Resource Broker for Service Oriented P2P/Grid Computing, International Workshop on Live Demonstrations of Grid Technologies and Applications (GridDemo 2001), In Conjunction with HPC Asia 2001, 26 September 2001, old Coast, Australia.
The Nimrod-G and Virtual Laboratory Toolset for Distributed Molecular Modelling for Drug Design on the Grid, The 2nd International Workshop on Live Demonstrations of Grid Technologies and Applications (GridDemo 2002), In conjuction with IEEE CCGrid 2002 symposium, May 21-24, 2002, Berlin, Germany.