The Norwegian research network
  
  Search:

Caching our Desire

DESIRE logo
Development of a European Service for Information on Research and Education

Plan for webcaching activities

This project is implemented by SURFnet, Utrecht University and UNINETT. DESIRE is a project in EU 4th Framework Programme.

This project aims to reduce network traffic due to WWW on a national level, and to improve access times for the end users. To achieve this it will implement both a local and a national web-cache service. This service which will use a 2 level network of web-cache servers strategically placed in the national network topology. A successful national cache service will also require a high degree of availability in the top-level server.

Tasks

Generic tasks

Continuing effort

  1. Project adminstration
  2. Publicity
  3. Tracking international development (HTTP 1.1)

T4.1 Survey of Caching Requirements and Alternatives

February 96 - April 96

  1. Evaluation of clients and servers
  2. Requirements for servers
  3. Specification for clients and servers

T4.2 Demonstrator for Caching Schemes

April 96 - March 97

  1. Web-cache server (local level)
  2. Web-cache server (national top-level)
  3. Testing new server-software
  4. Distribution of server-software
  5. Samsoft web-cache server package
  6. Documenting how to configure web-clients

T4.3 Validation of Caching Schemes

June 96 - June 97

  1. Software for analyzing web-cache logs
  2. Simulations
  3. Measuring network traffic
  4. Service roll-out

T4.4 Cost/Benefit Analysis Report

May 97 - July 97

  1. Cost-benefit report

T4.5 Validation of Caching Schemes (extended mesh)

May 97 - December 97

  1. Further testing with different caching mesh topologies
  2. Testing ISP-wide and inter-ISP mesh, parameter and configurations
  3. Caching server for end user clients
  4. Service roll-out

Deliverables and Milestones

This project plan is scheduled for revision late 1996, new version will be ready by December 1996 as a result of evaluation and input from users.

Revision made February 1997.

Generic tasks

Continuing effort: February 96 - July 97

  1. Project adminstration
  2. Publicity
  3. Tracking international development
Project adminstration
Co-ordination of the webcaching subtask and liason with Desire-project
Run by UNINETT
Publicity
This project needs some publicity to attract the initial test group members. Results needs to be disseminated.
Run by UNINETT, Utrecht University and SURFnet
Tracking international development
Main efforts in 1996 will be in tracking development of HTTP 1.1
Run by UNINETT, Utrecht University and SURFnet
Liason with Terena TF-CACHE
Contributing in the Terena taskforce on caching

T4.1 Survey of Caching Requirements and Alternatives

February 96 - April 96

  1. Evaluation of clients and servers
  2. Requirements for servers
  3. Specification for clients and servers
Evaluation of clients and servers

Run by UNINETT, Utrecht University and SURFnet
Requirements for servers

Run by UNINETT, Utrecht University and SURFnet
Specification for clients and servers

Run by UNINETT, Utrecht University and SURFnet

T4.2 Demonstrator for Caching Schemes

April 96 - March 97

  1. Web-cache server (local level)
  2. Web-cache server (national top-level)
  3. Testing new server-software
  4. Distribution of server-software
  5. Samsoft web-cache server package
  6. Documenting how to configure web-clients
Web-cache server (local level)

Run by Utrecht University
Web-cache server (national top-level)
The central UNINETT web-cache server will act as the top-level server in the cache chain. This server will be placed as close as possible to the network connection out of the country. This should maximize the bandwidth savings on this line.

The server software used today (CERN httpd) has some limitations. The chaining of several servers is very static, and the lower level servers will not work if the top-lever server becomes unavailable. This will hopefully be better in the next release, but as of now the top level server must have a high degree of availability to be accepted by the users.

The web-cache servers must be available at all times. To ensure this we need to monitor the servers by cron-jobs or other means. This monitoring should send messages by syslog to a central machine. This machine should then decide whom and how to warn. This monitoring will be implemented into the existing framework for the UNINETT FMS (national network operation center) service.
Run by UNINETT and SURFnet

Testing new server-software
We need to test different implementations, and possibly use it on the top-level server. We'll also keep our eyes open for other implementations, and test them when they become available.
Run by UNINETT, Utrecht University and SURFnet
Distribution of server-software
Develop configuration for server-software and offer a package for distribution
Run by UNINETT and Utrecht University
Samsoft web-cache server package
Making an easy to install SAMSOFT cache-server package of the web-server software, and documenting how this server should be installed and configured. The participating UNINETT members should install and run this software on their own, but they will get technical support when necessary. UNINETT will also make a recommendation on how the web-cache server and it's clients should be configured to cache. To do this one must know the local network topology. The participating UNINETT members will have to provide as much disk as they can afford.

UNINETT members with other server solutions (HP, SGI, SUN, DEC or Linux) will get assistance in setting up a web-cache server.
Run by UNINETT

Documenting how to configure web-clients
There is a lot of different web-clients out there. Most of them have support for using web-cache servers. This document will describe how different web-clients can be configured to use a web-server. Participating institutions will also need some assistance in figuring out what to cache/not cache according to the local network topology.
Run by UNINETT, Utrecht University and SURFnet

T4.3 Validation of Caching Schemes

June 96 - June 97

  1. Software for analyzing web-cache logs
  2. Simulations
  3. Measuring network traffic
  4. Service roll-out
Software for analyzing web-cache logs
This software is needed to verify the success of the cache-network. It will measure cache hits in bytes and number of connections. It will also measure the number of users, URLs served, distribution of file sizes, disk/inode usage, machine load and distribution of protocols cached.
Run by UNINETT
Simulations
In order to experiment with different configurations, cache-TTL, mesh building, and placing of webcaching servers in the network, simulations will be run to provide data on optimal solutions.
Run by UNINETT
Measuring network traffic
IP network management and developpement project will be involved in monitoring the web-traffic before and after a web-cache server is introduced. This, and the software for analyzing web-cache logs, will be a part of the verification process.
Run by UNINETT
Service roll-out
Spreading the use of webcaching employing the experiences gained.
Run by SURFnet and UNINETT

T4.4 Cost/Benefit Analysis Report

May 97 - July 97

  1. Cost-benefit report

A report on the cost-benefit of introducing caching in a local and national network.
Run by SURFnet

This task will be further specified by the end of 1996

T4.5 Validation of Caching Schemes (extended mesh)

May 97 - December 97

  1. Further testing with different caching mesh topologies
  2. Testing ISP-wide and inter-ISP mesh, parameter and configurations
  3. Caching server for end user clients
  4. Service roll-out
Further testing with different caching mesh topologies
Network topology is important for the cache mesh, and further investigation is needed
Run by UNINETT and SURFnet

Testing ISP-wide and inter-ISP mesh, parameter and configurations
Extended testing of ISP-wide configuration of mesh, expiration schemes.
Run by UNINETT and SURFnet

Caching server for end user clients
SURFnet establishes a caching server for end user clients, for use by institutions who are not able to run their own web cache server. Characteristics for a first level cache differs from a upper level web cache.
Run by SURFnet

Service roll-out
Continuing effort from T4.3 in extended mesh. Spreading the use of webcaching employing the experiences gained.
Run by UNINETT and SURFnet

Desire | SURFnet | UNINETT

Ingrid.Melve@uninett.no 2004-05-13