Genome Networking

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
Click the link to see some [[sample-ps-sites|sample sites]] and their perfsonar ping latency graphs.\\
If Smokeping needs to be installed and configured, [[http://www.howtoforge.com/monitoring_network_latency_smokeping_debian_etch|this guide]] is what was used to install it on Balrog.\\ It is for an older version of Debian, but it still works for the newest version (Squeeze).

This site documents the work evaluating technologies for transferring genomes to the CBSE-managed TCGA archive site.

For current status see:
{{:genomepaper-6-22-11.pdf|LATEST REPORT}}
{{:genomereport-6-22.tar.gz|LATEST REPORT FILES}}
{{:paper.pdf|REPORT-1st-draft}}.

Recent presentation on this project, made for CE-280N  {{:ce-280n_presentation.ppt|LATEST PRESENTATION}}.

The plan for this evaluation site is as follows:

  * Setup a [[http://www.perfsonar.net|perfSONAR]] node at UCSC to monitor paths to TCGA-related destinations around the Internet.
  * Setup a testbed using the [[http://info.iet.unipi.it/~luigi/dummynet/|Dummynet]] tool that simulates queue and bandwidth limitations, delays, packet losses, and multipath effects.

Additional reference information can be found at:

  * ESnet's [[http://fasterdata.es.net/|fasterdata]] site.
    * An interesting [[http://fasterdata.es.net/assets/fasterdata/Dart-tutorial.pdf|presentation]] on this site at the Internet2 Joint Techs conference in 2010 by Eli Dart of ESnet.
  * Pittsburgh Supercomputing Center's site on [[http://www.psc.edu/networking/projects/tcptune/|Enabling High Performance Data Transfers]].
  * [[http://www.web100.org/|Web100 Project]]
  * Presentation on the [[http://www.es.net/assets/Uploads/20110201-dart-science-dmz.pdf|Science DMZ]] concept.

The 3 tools we are planning the test are:
^ Tool      ^ Installation ^  Results ^
| [[http://globus.org/toolkit/docs/3.2/gridftp/|GridFTP]]    | [[GridFTPInstall|Installation]]     | [[GridResults|Results]] |
| [[http://monalisa.cern.ch/FDT/|Faster Data Transfer]]    | [[FDTInstall|Installation]]| [[FDTResults|Results]] |
| [[http://genome.ucsc.edu/admin/git.html|paraFetch]]    | [[paraFetch|Installation]]     | [[ParaResults|Results]] |

Experiment Details:
  - [[ScenarioProposal| Scenario Proposal]]
  - [[Locations | Remote Locations]]
  - [[TcpTuning | TCP Tuning]]
  - **perfSonar Sites:** [[http://ndb1.internet2.edu/perfAdmin/directory.cgi|Directory]]

^ Sites      ^ PerfSONAR ^
| Sequencing Center at Washington University, St Louis    | [[WUperfSonar|Results]]     | 
| Sequencing Center at Baylor College of Medicine, Houston    | [[BCperfSonar|Results]]|
| Broad Institute, Boston    | [[BIperfSonar|Results]]     |
| University of North Carolina Sequencing Center    | [[UNCperfSonar|Results]]     |
| University Of British Columbia Sequencing Center    | [[UBCperfSonar|Results]]     |

**Data sites:**
    * [[http://hgdownload.cse.ucsc.edu/downloads.html|UCSC Genome data download page]]
    * [[ftp://hgdownload.cse.ucsc.edu/goldenPath/|FTP site:UCSC Genome data]]

Related research:
  - [[http://bioftp.org/index.php/Korean_Reference_Genome_data| KOREF Human Genome]]
  - [[http://portal.acm.org/citation.cfm?id=1592576|VL2: A Scalable and Flexible Data Center Network]] : [[VL2Notes|Notes]]
  - [[http://portal.acm.org/citation.cfm?id=956993.957002&coll=DL&dl=ACM&CFID=16657217&CFTOKEN=91943640|Web100: Extended TCP Instrumentation]]: [[Web100|Notes]]
  - [[http://research.microsoft.com/en-us/um/people/maheshba/papers/thesis.pdf|RELIABLE COMMUNICATION FOR DATACENTERS]]: [[DCNotes|Notes]]
  - [[http://nar.oxfordjournals.org/content/early/2011/01/25/nar.gkr009.full.pdf+html|Genome Compression]] : [[GCNotes|Notes]]
  - TCP Variants:
  * [[http://www4.ncsu.edu/~rhee/export/bitcp/cubic-paper.pdf|TCP Cubic Variant]]: [[Cubic|Notes]]
  * [[ftp://ftp.cs.arizona.edu/xkernel/Papers/vegas.ps|TCP Vegas]] : [[Vegas|Notes]] 
  * [[http://gridftp.bio-mirror.net/biomirror/|Bio Mirror (uses GridFTP + UDT)]]
  * [[http://www.ogf.org/OGF22/materials/1144/gridftp-ogf22.ppt|Open Grid Forum - Presentation on GridFTP]]
  * GridFTP:
    * [[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.97.8734&rep=rep1&type=pdf| The Globus Striped GridFTP Framework and Server]]
    * [[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.109.1941&rep=rep1&type=pdf|Globus GridFTP: What’s New in 2007]]
    * [[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.138.6285&rep=rep1&type=pdf|The Globus eXtensible Input/Output System (XIO)]]
    * [[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.58.4783&rep=rep1&type=pdf| Internet Congestion Control for Future High Bandwidth-Delay Product Environments]]
  * UDT:
    * [[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.113.5963&rep=rep1&type=pdf| UDT: UDP-based Data Transfer for High-Speed Wide Area Networks ]]
Other links:
  * [[http://www.dokuwiki.org/syntax|Wiki Syntax]]
Genome Networking

User Tools

Site Tools

Page Tools
