I think tomorrow is the promised "Monday after PyCon" ;) Sorry to pester, but just wanted you to know that there's an audience for your cluster management code. Thanks!
Peter Skomoroch
· 1 year ago
Patrick,
No problem, the cluster management code is checked in now at google code:
http://code.google.com/p/elasticwulf/
You can grab it from the subversion repository with the following command:
Thank you for posting this. I am going to try this out this weekend or next week. I need to install R and snowfall/sfCluster and rebundle, and this should fit my needs just fine. I am going to try some simulation for solving stochastic differential equations. This is just the ticket.
Quick question: Is there a particular reason why you chose fedora core 6 as opposed to a newer version? I am more of an Ubuntu guy so I don't know the difference in the Fedora versions. Would it be worth trying to recreate the environment in a more recent version?
Peter Skomoroch
· 10 months ago
Stephen: Back when I first created the AMI images (late 2006?) Fedora was the most stable base image on EC2. Most of the MPI/NFS docs I ran across were also Fedora / Red Hat based, so it was the easiest option at the time. I'm working with the infochimps.org guys on a new swiss army knife AMI based on Ubuntu, and I'm planning on migrating the elasticwulf launch utilities over as well to take better advantage of boto and EBS data volumes. Stay tuned.
Stephen
· 10 months ago
Thanks for the quick response. Let me know if I can help with testing or anything. This is really cool stuff.
All Comments
I think tomorrow is the promised "Monday after PyCon" ;) Sorry to pester, but just wanted you to know that there's an audience for your cluster management code. Thanks!
Patrick,
No problem, the cluster management code is checked in now at google code:
http://code.google.com/p/elasticwulf/
You can grab it from the subversion repository with the following command:
svn checkout http://elasticwulf.googlecode.com/svn/trunk/ elasticwulf-read-only
Some people have run into an issue where the configuration script asks for a password, I'll try to track down what is causing it and check in a fix.
In the meantime, there is a workaround in the comments of the old tutorial:
http://www.datawrangling.com/mpi-cluster-with-python-and-amazon-ec2-part-2-of-3.html#comment-1148
-Pete
Thank you for posting this. I am going to try this out this weekend or next week. I need to install R and snowfall/sfCluster and rebundle, and this should fit my needs just fine. I am going to try some simulation for solving stochastic differential equations. This is just the ticket.
Quick question: Is there a particular reason why you chose fedora core 6 as opposed to a newer version? I am more of an Ubuntu guy so I don't know the difference in the Fedora versions. Would it be worth trying to recreate the environment in a more recent version?
Stephen: Back when I first created the AMI images (late 2006?) Fedora was the most stable base image on EC2. Most of the MPI/NFS docs I ran across were also Fedora / Red Hat based, so it was the easiest option at the time. I'm working with the infochimps.org guys on a new swiss army knife AMI based on Ubuntu, and I'm planning on migrating the elasticwulf launch utilities over as well to take better advantage of boto and EBS data volumes. Stay tuned.
Thanks for the quick response. Let me know if I can help with testing or anything. This is really cool stuff.
Steve:
I'm looking at resurrecting this on Ubuntu now, just want to point readers to your site since I get a lot of questions about running distributed Matlab on EC2: Econ Steve: distributed matlab (part 1 of 3) - compiling.
One of the posts I'm working on is replicating this microsoft risk calculation example using open source alternatives on Amazon EC2: http://blog.jonudell.net/2008/03/27/cluster-computing-with-large-data-for-the-classroom/
-Pete